Electronic device for tracking objects

ABSTRACT

Systems, methods, and non-transitory media are provided for tracking operations using data received from a wearable device. An example method can include determining a first position of a wearable device in a physical space; receiving, from the wearable device, position information associated with the wearable device; determining a second position of the wearable device based on the received position information; and tracking, based on the first position and the second position, a movement of the wearable device relative to an electronic device.

TECHNICAL FIELD

The present disclosure generally relates to tracking systems and, morespecifically, an electronic device for tracking objects (e.g., forextended reality operations and/or power savings) using data from awearable device.

BACKGROUND

Extended reality (e.g., augmented reality, virtual reality, etc.)devices, such as smart glasses and head-mounted displays (HMDs),generally implement cameras and sensors to track the position of theextended reality (XR) device and other objects within the physicalenvironment. The XR reality devices can use the tracking information toprovide a user of the XR device a realistic XR experience. For example,an XR device can allow a user to experience or interact with immersivevirtual environments or content. To provide realistic XR experiences, XRtechnologies can integrate virtual content with the physical world,which can involve matching the relative pose and movement of objects anddevices. The XR technologies can use tracking information to calculatethe relative pose of devices, objects, and/or maps of the real-worldenvironment in order to match the relative position and movement of thedevices, objects, and/or the real-world environment, and anchor contentto the real-world environment in a convincing manner. The relative poseinformation can be used to match virtual content with the user'sperceived motion and the spatio-temporal state of the devices, objects,and real-world environment.

BRIEF SUMMARY

Disclosed are systems, methods, and computer-readable media for trackingobjects and controlling operations, states, and/or settings of anelectronic device. For example, an electronic device can use/leveragedata from a wearable device to implement tracking, power savings, and/orvarious operations (e.g., extended reality operations, etc.). Theelectronic device can communicate with a wearable device to obtain datafrom the wearable device. In some examples, the wearable device canassist the electronic device with tracking, power savings, and/or otheroperations/settings. According to at least one example, a method isprovided for tracking operations using data received from a wearabledevice. The method can include determining a first position of awearable device in a physical space; receiving, from the wearabledevice, position information associated with the wearable device;determining a second position of the wearable device based on thereceived position information; and tracking, based on the first positionand the second position, a movement of the wearable device relative tothe electronic device.

According to at least one example, a non-transitory computer-readablemedium is provided for tracking operations using data received from awearable device. The non-transitory computer-readable medium can includeinstructions stored thereon which, when executed by one or moreprocessors, cause the one or more processors to: determine a firstposition of a wearable device in a physical space; receive, from thewearable device, position information associated with the wearabledevice; determine a second position of the wearable device based on thereceived position information; and track, based on the first positionand the second position, a movement of the wearable device relative tothe electronic device.

According to at least one example, an apparatus is provided for trackingoperations using data received from a wearable device. The apparatus caninclude memory and one or more processors coupled to the memory, the oneor more processors being configured to: determine a first position of awearable device in a physical space; receive, from the wearable device,position information associated with the wearable device; determine asecond position of the wearable device based on the received positioninformation; and track, based on the first position and the secondposition, a movement of the wearable device relative to the electronicdevice.

According to at least one example, another apparatus is provided fortracking operations using data received from a wearable device. Theapparatus can include means for determining a first position of awearable device in a physical space; receiving, from the wearabledevice, position information associated with the wearable device;determining a second position of the wearable device based on thereceived position information; and tracking, based on the first positionand the second position, a movement of the wearable device relative tothe electronic device.

In some aspects, the method, non-transitory computer-readable medium,and apparatuses described above can determine, based on the secondposition of the wearable device and/or the tracked movement of thewearable device, whether the wearable device is within a field-of-view(FOV) of one or more image sensors on the electronic device and/orvisible to the one or more image sensors on the electronic device. Insome aspects, the method, non-transitory computer-readable medium, andapparatuses described above can track, based on the second position ofthe wearable device and/or the tracked movement of the wearable device,a location of a hand associated with the wearable device. In someexamples, the hand can include a hand wearing the wearable device (e.g.,on a wrist, on a finger, etc.) or holding the wearable device.

In some examples, determining the first position of the wearable devicecan include receiving, from the wearable device, image data from one ormore image sensors on the electronic device and/or data associated withone or more measurements from one or more sensors on the wearabledevice; and determining the first position of the wearable device basedon the image data from the one or more image sensors and/or dataassociated with the one or more measurements from the one or moresensors.

In some examples, the data can include a distance of the wearable devicerelative to one or more objects (e.g., a wall, a door, furniture, adevice, a person, an animal, etc.), a velocity vector indicating avelocity of movement of the wearable device, a touch signal measured bya touch sensor from the one or more sensors, audio data from an audiosensor from the one or more sensors, and/or an elevation of the wearabledevice in the physical space. In some cases, the one or more objects caninclude the electronic device, a body part (e.g., a hand, a leg, an arm,a head, a torso, etc.) associated with a user of the wearable device,and/or an input device (e.g., a controller, a keyboard, a remote, etc.).

In some examples, the position information can include a pose of thewearable device. In some cases, the position information can be based onsensor data from one or more sensors on the wearable device. In somecases, the position information can include a measurement from aninertial measurement unit from one or more sensors on the wearabledevice and/or an elevation measured by a pressure sensor from the one ormore sensors.

In some cases, the second position can include a position of thewearable device relative to the electronic device. In some cases, thesecond position can include a position of the wearable device within acoordinate system, such as a coordinate system of the wearable deviceand/or a coordinate system of the electronic device.

In some aspects, tracking the movement of the wearable device caninclude determining the first position of the wearable device within afirst coordinate system of the wearable device; transforming the firstcoordinate system of the wearable device to a second coordinate systemof the electronic device; and determining the second position of thewearable device within the second coordinate system of the electronicdevice.

In some cases, the wearable device can include a bracelet, a ring, or aglove.

In some aspects, the method, non-transitory computer-readable medium,and apparatuses described above can capture, based on a determinationthat the wearable device is within the FOV of the one or more imagesensors and/or visible to the one or more image sensors, one or moreimages of the hand via at least one image sensor from the one or moreimage sensors; and track the location of the hand based on the one ormore images of the hand. In some examples, the location of the hand istracked relative to a first coordinate system of the wearable device.

In some aspects, the method, non-transitory computer-readable medium,and apparatuses described above can determine, based on the positioninformation, that the wearable device is outside of the FOV of the oneor more image sensors and moving towards an area within the FOV of theone or more image sensors; and based on the determining that thewearable device is outside of the FOV of the one or more image sensorsand moving towards the area within the FOV of the one or more imagesensors, initiate one or more imaging operations and/or one or moretracking operations at the electronic device. In some examples, the oneor more tracking operations can be at least partly based on image datafrom the one or more imaging operations.

In some aspects, the method, non-transitory computer-readable medium,and apparatuses described above can adjust, based on a firstdetermination that the wearable device is within a first FOV of a firstimage sensor on the electronic device and/or a second determination thatthe wearable device is visible to the first image sensor on theelectronic device, a first setting of the first image sensor. In somecases, the first setting can include a power mode of the first imagesensor and/or an operating state of the first image sensor. In someaspects, the method, non-transitory computer-readable medium, andapparatuses described above can adjust, based on a third determinationthat the wearable device is outside of a second FOV of a second imagesensor on the electronic device and/or a fourth determination that thewearable device is not visible to the second image sensor on theelectronic device, a second setting of the second image sensor. In someexamples, the second setting can include a power mode of the secondimage sensor and/or an operating state of the second image sensor.

In some examples, adjusting the first setting of the first image sensorcan include changing the power mode of the first image sensor from afirst power mode to a second power mode including a higher power modethan the first power mode and/or changing the operating state of thefirst image sensor from a first operating state to a second operatingstate including a higher operating state than the first operating state.In some examples, the second operating state can include a higherframerate and/or a higher resolution.

In some examples, adjusting the second setting of the second imagesensor can include changing the power mode of the second image sensorfrom a first power mode to a second power mode including a lower powermode than the first power mode and/or changing the operating state ofthe second image sensor from a first operating state to a secondoperating state including a lower operating state than the firstoperating state. In some cases, the second operating state can include alower framerate and/or a lower resolution.

In some aspects, the method, non-transitory computer-readable medium,and apparatuses described above can track, in response to adetermination that the wearable device is outside of the FOV of the oneor more image sensors on the electronic device and/or a view of the oneor more image sensors to the wearable device is obstructed by one ormore objects, a location of the wearable device based on additionalposition information from the wearable device. In some aspects, themethod, non-transitory computer-readable medium, and apparatusesdescribed above can track, in response to a determination that thewearable device is within the FOV of the one or more image sensors but aview of the one or more image sensors to the wearable device isobstructed, a location of the wearable device based on additionalposition information from the wearable device.

In some aspects, the method, non-transitory computer-readable medium,and apparatuses described above can initialize, in response to thedetermination that the wearable device is within the FOV of the one ormore image sensors but the view of the one or more image sensors to thewearable device is obstructed, the one or more image sensors.

In some aspects, the method, non-transitory computer-readable medium,and apparatuses described above can receive, from the wearable device,an input configured to trigger a privacy mode at the electronic device;and based on the input configured to trigger the privacy mode, adjust anoperating state of one or more image sensors at the electronic device toan off state and/or a disabled state. In some examples, the input can bebased on sensor data from one or more sensors on the wearable device. Insome cases, the sensor data can indicate a touch signal corresponding toa touch input at the wearable device, a location of the wearable device,and/or a distance between the wearable device and a body part of a userof the wearable device.

In some aspects, the method, non-transitory computer-readable medium,and apparatuses described above can receive, from the wearable device,an additional input configured to trigger the electronic device to stopthe privacy mode. In some cases, the additional input can be based onsensor data indicating a touch signal corresponding to a touch input atthe wearable device, a location of the wearable device corresponding toa location of a body part of a user of the wearable device, and/or aproximity between the wearable device and the body part.

In some aspects, the method, non-transitory computer-readable medium,and apparatuses described above can determine, based on data from thewearable device and/or a command from the wearable device, one or moreextended reality (XR) inputs to an XR application on the electronicdevice. In some examples, the one or more XR inputs can include amodification of a virtual element along multiple dimensions in space, aselection of the virtual element, a navigation event, and/or a requestto measure a distance defined by the first position of the wearabledevice, the second position of the wearable device, and/or the movementof the wearable device.

In some examples, the virtual element can include a virtual objectrendered by the electronic device, a virtual plane in an environmentrendered by the electronic device, and/or the environment rendered bythe electronic device. In some examples, the navigation event caninclude scrolling rendered content and/or moving from a first interfaceelement to a second interface element.

In some aspects, the method, non-transitory computer-readable medium,and apparatuses described above can receive, from the wearable device,an input configured to trigger an adjustment of one or more XRoperations at the electronic device. In some examples, the one or moreXR operations can include object detection, object classification,object tracking, pose estimation, and/or shape estimation. In someexamples, the one or more sensors can include at least one of anaccelerometer, a gyroscope, a pressure sensor, an audio sensor, a touchsensor, and a magnetometer.

In some aspects, the apparatuses described above can include one or moresensors. In some aspects, the apparatuses described above can include awearable ring. In some aspects, an apparatus described above can includea mobile device. In some examples, the apparatuses can include a handcontroller, a mobile phone, a wearable device, a display device, amobile computer, a head-mounted device, and/or a camera.

This summary is not intended to identify key or essential features ofthe claimed subject matter, nor is it intended to be used in isolationto determine the scope of the claimed subject matter. The subject mattershould be understood by reference to appropriate portions of the entirespecification of this patent, any or all drawings, and each claim.

The foregoing, together with other features and embodiments, will becomemore apparent upon referring to the following specification, claims, andaccompanying drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

In order to describe the manner in which the various advantages andfeatures of the disclosure can be obtained, a more particulardescription of the principles described above will be rendered byreference to specific embodiments thereof, which are illustrated in theappended drawings. Understanding that these drawings depict only exampleembodiments of the disclosure and are not to be considered to limit itsscope, the principles herein are described and explained with additionalspecificity and detail through the use of the drawings in which:

FIG. 1 is a diagram illustrating an example of an extended realitysystem and a wearable device used for extended reality experiences andfunctionalities, in accordance with some examples of the presentdisclosure;

FIGS. 2A and 2B illustrates examples of a ring device worn on a fingerof a user interacting with an extended reality system, in accordancewith some examples of the present disclosure;

FIG. 3 is a diagram illustrating an example process for integrating datafrom a wearable device for extended reality operations at an extendedreality system, in accordance with some examples of the presentdisclosure;

FIG. 4 is a flow diagram illustrating an example process for using awearable device with an extended reality system (e.g., for enhancedtracking, power savings, and privacy functionalities), in accordancewith some examples of the present disclosure;

FIG. 5A is a flow diagram illustrating an example process for using awearable device with an extended reality device, in accordance with someexamples of the present disclosure;

FIG. 5B is a flow diagram illustrating an example process for trackingobjects, in accordance with some examples of the present disclosure; and

FIG. 6 illustrates an example computing device architecture, inaccordance with some examples of the present disclosure.

DETAILED DESCRIPTION

Certain aspects and embodiments of this disclosure are provided below.Some of these aspects and embodiments may be applied independently andsome of them may be applied in combination as would be apparent to thoseof skill in the art. In the following description, for the purposes ofexplanation, specific details are set forth in order to provide athorough understanding of embodiments of the application. However, itwill be apparent that various embodiments may be practiced without thesespecific details. The figures and description are not intended to berestrictive.

The ensuing description provides example embodiments only, and is notintended to limit the scope, applicability, or configuration of thedisclosure. Rather, the ensuing description of the exemplary embodimentswill provide those skilled in the art with an enabling description forimplementing an exemplary embodiment. It should be understood thatvarious changes may be made in the function and arrangement of elementswithout departing from the spirit and scope of the application as setforth in the appended claims.

As previously mentioned, extended reality (e.g., augmented reality,virtual reality, etc.) devices, such as smart glasses and head-mounteddisplays (HMDs), can implement cameras and sensors to track the positionof the extended reality (XR) device and other objects within thephysical environment. The XR reality devices can use such trackinginformation to provide a user of the XR device a realistic XRexperience. For example, an XR device can allow a user to experience orinteract with immersive virtual environments or content. To providerealistic XR experiences, XR technologies can integrate virtual contentwith the physical world. The XR technologies can use trackinginformation to calculate the relative pose of devices, objects, and/ormaps of the real-world environment in order to match the relativeposition and movement of the devices, objects, and/or the real-worldenvironment, and anchor content to the real-world environment in aconvincing/realistic manner. The relative pose information can be usedto match virtual content with the user's perceived motion and thespatio-temporal state of the devices, objects, and real-worldenvironment.

In some examples, an XR device can implement a tracking algorithm thatuses sensor data to track the position of an object in three-dimensional(3D) space, such as a hand, a finger, an input device (e.g., acontroller, a stylus, a joystick, a glove, etc.). For example, atracking algorithm can use measurements from various sensors, such asimage sensors and inertial measurement units (IMUs), a pose of acontroller (or data/measurements thereof), and/or a motion model for auser's hand(s) to estimate a hand/controller pose used by the XR deviceduring an XR experience. In some cases, the tracking algorithm can usesensor data to predict the location of a user's hand(s), a location of acamera of the XR device, and/or a state of the XR device. The XR devicecan measure the location of the XR device, camera, and/or user'shand(s), and use such measurement to update the state of the XR device.

Often, an XR device may implement multiple cameras for trackingrobustness. For example, an XR device can implement multiple cameras totrack a full range of motion (e.g., up, down, left, right, etc.) of ahand (and/or any other object). Together, the multiple cameras canprovide a full or larger field-of-view (FOV) in 3D space to capture (andtrack) an object from different relative positions in 3D space. Whilethe multiple cameras implemented by the XR device and the tracking,detection, classification, etc., operations performed by the XR devicecan significantly aid in tracking, detection, and classificationoperations, they also use a significant amount of power and computeresources at the XR device. This can negatively impact the performanceand the more-limited battery life of the XR device.

In some cases, the cameras used by XR devices can also create privacyissues. For example, in some situations, a user of an XR device may notwant cameras on the XR device to capture images in a particularscene/setting or to even be turned on. In such cases, the user may wantto turn off or disable the use of cameras at the XR device. However,turning off or disabling the use of cameras at the XR device cansignificantly limit the XR device's ability to track/detect objects oreven prevent the XR device from tracking/detecting objects, which cansignificantly impact or prevent XR functionalities/experiences at the XRdevice.

In some aspects, systems, apparatuses, processes (also referred to asmethods), and computer-readable media (collectively referred to hereinas “systems and techniques”) are described herein for using a wearableaccessory (or wearable device) that interfaces/interacts with an XRdevice to aid in tracking, provide power savings, and/or increase userprivacy during XR experiences. In some examples, a wearable accessorycan be worn by a user of an XR device during an XR experience. Thewearable accessory can be worn on a user's finger (or multiple fingers),a wrist, an ankle, and/or any other body part. The wearable accessorycan include embedded sensors configured to obtain measurements of astate (e.g., a position, movement, etc.) of the wearable accessory in 3Dspace, and thus the state of the body part on which the wearableaccessory is worn (e.g., a finger, a hand, etc.). The wearable accessorycan provide the measurements to the XR device, which can integrate withits tracking system for more robust tracking and accuracy.

The wearable accessory can include, for example and without limitation,a ring that can be worn on a finger, a sleeve of rings that can be wornon multiple fingers, a bracelet that can be worn on a wrist, a glovethat can be worn on a hand, etc. For example, the wearable accessory caninclude a ring device worn on a user's finger. The ring device caninclude embedded sensors that capture tracking measurements (e.g.,position measurements, motion measurements, etc.) of the ring device(and thus the position of the user's finger/hand). The ring device cansend the tracking measurements to an XR device of the user, which canuse the tracking measurements (with or without measurements separatelyobtained by the XR device) to track, detect, and/or classify the user'sfinger/hand. In some examples, the tracking measurements from the ringdevice can help the XR device to track the user's finger/hand even whenthe user's finger/hand is outside (or partially outside) of a FOV of oneor more cameras at the XR device, when lighting conditions in the sceneprevent or limit the ability of the XR device to detect and/or image theuser's finger/hand, when a view of the one or more cameras to the user'sfinger/hand is occluded by something, etc.

In some cases, the wearable accessory can help reduce power consumptionat the XR device by triggering the XR device to modify an operation ofthe cameras at the XR device and/or a tracking system used by the XRdevice. For example, if the wearable accessory is outside of a FOV ofone or more cameras on the XR device or if a view of the one or morecameras to the wearable accessory is obstructed (e.g., by an object,poor lighting conditions, etc.), the XR device can use trackingmeasurements from the wearable accessory to reduce a processing mode ofthe one or more cameras (e.g., turn off the one or more cameras, reducea framerate of the one or more cameras, reduce operations by the one ormore cameras, and/or otherwise reduce power consumption by the one ormore cameras) to avoid unnecessary use of and/or power consumption bythe one or more cameras. The XR device can use the tracking measurementsto track the wearable accessory and/or aid in tracking the wearableaccessory while the one or more cameras are in the reduced processingmode. In some examples, the XR device can use the tracking measurementsto track the wearable accessory with or without other sensor data suchas, for example, image data from the one or more cameras in the reducedprocessing mode and/or image data from one or more other cameras of theXR device.

To illustrate, if in a non-limiting example the XR device implements 4cameras and the wearable accessory is outside of the FOV of 3 of thecameras, in order to reduce power consumption, the XR device can turnoff (or otherwise reduce a processing mode of) the 3 cameras while suchcameras are unable to capture tracking images of the wearable accessory(and the ring/hand or other body part wearing the wearable accessory).The XR device can use image data from the remaining camera and trackingmeasurements from the wearable accessory to track, detect, and/orclassify user's hand/finger wearing the wearable accessory (or any otherbody part wearing the wearable accessory), while the other 3 cameras areturned off (or in a particular reduced processing mode). The trackingmeasurements from the wearable accessory can help increase the trackingfidelity/accuracy of the XR device, and can trigger subsequentprocessing mode adjustments of any of the cameras of the XR device basedon a position and/or motion of the wearable accessory. If all 4 camerasof the XR device are unable to capture an image of the wearableaccessory, the XR device can use the tracking measurements from thewearable accessory to continue tracking the wearable accessory (and thebody part(s) on which it is worn). The number of cameras (e.g., 4) inthis example is merely one illustrative example provided for explanationpurposes and should not be construed as limiting. One of ordinary skillin the art will recognize from this disclosure that, in other examples,the XR device can implement any other number of cameras.

In some examples, the XR device can use the tracking measurements fromthe wearable accessory to determine where the wearable accessory (andthe body part(s) wearing the wearable accessory) are and/or will belocated and reduce a processing mode (e.g., turn off, reduce aframerate, etc.) of a camera(s) when the wearable accessory is outsideof a coverage area of the camera(s) for a certain period of time. Thetracking measurements from the wearable accessory can also triggerreactivation and/or a higher/full processing mode of the camera(s) whenthe wearable accessory is again within the coverage area of thecamera(s).

In some cases, the tracking measurements from the wearable accessory canhelp the XR device track the wearable accessory and detect a handwearing the wearable accessory (or any other body part) during difficultimaging conditions such as poor lighting conditions, clutteredbackgrounds, etc. In some examples, the tracking measurements from thewearable accessory can help the XR device distinguish between differentobjects in a scene. For example, tracking measurements from a wearableaccessory worn on a finger can help the XR device distinguish betweenthe hand of the finger wearing the wearable accessory and other hands inthe scene. To illustrate, the tracking measurements from the wearableaccessory can provide the XR device a reference of which hand to track.The XR device can use this information to track the hand of the fingerwearing the wearable accessory as the XR device can determine which ofvarious hands in the scene is the hand intended to be tracked.

In some cases, the XR device can use one or more tracking measurementsfrom the wearable accessory to distinguish an object of interest (e.g.,a hand, a finger, etc.) associated with the wearable accessory fromother objects (e.g., different body parts, devices, people, etc.) toallow the XR device to detect and track the object of interest (or thecorrect object). In some cases, the wearable accessory can send one ormore signals to the XR device with a certain frequency pattern. Thefrequency pattern can be associated with the wearable accessory and canbe used by the XR device to recognize the wearable accessory and theobject of interest associated with the wearable accessory. In someexamples, the XR device can use the frequency pattern to distinguish thewearable accessory and the object of interest from other objects in anenvironment. In some cases, the wearable accessory can include one ormore visual patterns that the XR device can detect from one or moreimages to recognize the wearable accessory in the environment.

In some cases, the tracking measurements from the wearable accessory canhelp the XR device save power by reducing tracking operations and/oroptimizing tracking operations. For example, the wearable accessory canobtain tracking measurements and provide a velocity vector to a handtracking engine of the XR device to the help predict where in an imagecaptured by a camera(s) of the XR device to search for a hand associatedwith the wearable accessory. The velocity vector can also help reducethe power consumed by the XR device in searching for the hand in animage by reducing the search time and/or search field. For example,using the tracking measurements from the wearable accessory, the XRdevice can search for the hand in one or more areas of an image withoutsearching the entire image. By reducing the search, the XR device canalso reduce the power consumed by the tracking algorithm (and theefficiency of the tracking algorithm) in finding the hand within theimage. The XR device can thus avoid searching unnecessary regions on theimage.

In some examples, the tracking measurements from the wearable accessorycan reduce a power consumption at the XR device by reducing and/oradjusting a tracking workflow implemented by the XR device. For example,the tracking measurements from the wearable accessory can indicate thata hand wearing the wearable accessory and tracked by the XR device isoutside of the FOV of the cameras of the XR device. The XR device canstop certain tracking operations, such as hand detection and/or handclassification operations, while the wearable accessory and the hand areoutside of the FOV of the cameras. In some examples, the XR device canreduce the number of tracking operations while the wearable accessoryand the hand are outside of the FOV of the cameras. For example, whenthe wearable accessory and the hand are outside of the FOV of thecameras, rather than performing hand detection and/or handclassification operations on every image captured by any of the cameras,the XR device can perform the hand detection and/or hand classificationoperations every n number of images.

In some cases, the wearable accessory can provide tracking measurementsto the XR device to allow the XR device to continue tracking thewearable accessory (and a body part wearing the wearable accessory) evenif the user of the XR device turns off the camera(s) on the XR devicefor privacy. Moreover, the wearable accessory can also be used totrigger a privacy mode at the XR device to turn off the camera(s) on theXR device. For example, the user can remove the wearable accessoryand/or provide an input to the wearable accessory (e.g., a touch input,a motion-based input, etc.) to trigger a privacy mode at the XR device.The XR device can interpret the removal of the wearable accessory and/orthe input to the wearable accessory as an input request to trigger aprivacy mode.

The wearable accessory can include one or more sensors to track a stateof the wearable accessory such as, for example, a position, a motion,etc. For example, the wearable accessory can include an inertialmeasurement unit (IMU) that can integrate multi-axes, accelerometers,gyroscopes, and/or other sensors to provide the XR device an estimate ofthe pose of the wearable accessory (and thus a body part wearing thewearable accessory) in physical space. The wearable accessory caninclude one or more sensors such as ultrasonic sensors and/ormicrophones used for ranging of the wearable accessory (and the bodypart wearing the wearable accessory). In some examples, one or moreultrasonic sensors and/or microphones can help determine if the wearabledevice is close to another object(s), if a user's hands (or other bodypart) are closer together or farther apart, if any of the user's hands(or other body part) are close to one or more other objects, etc. Insome examples, a barometric air pressure sensor in the wearableaccessory can determine relative elevation changes associated with thewearable accessory. The wearable accessory can send measurements fromone or more sensors to the XR device, which can use the sensormeasurements as further described herein.

The present technologies will be described in the following disclosureas follows. The discussion begins with a description of example systemsand techniques for using a wearable accessory with an XR device fortracking, privacy, and/or power savings, as illustrated in FIGS. 1through 4 . A description of an example process for using a wearabledevice with an XR device, as illustrated in FIG. 5 , will then follow.The discussion concludes with a description of an example computingdevice architecture including example hardware components suitable forperforming XR and associated operations, as illustrated in FIG. 6 . Thedisclosure now turns to FIG. 1

FIG. 1 is a diagram illustrating an example of an XR system 100 and awearable device 150 for XR experiences, in accordance with some examplesof the present disclosure. The wearable device 150 can represent awearable accessory used with the XR system 100 as further describedherein. The wearable accessory can include, for example and withoutlimitation, a ring, a bracelet, a ring sleeve, a glove, a watch, or anyother wearable device.

The XR system 100 and the wearable device 150 can be communicativelycoupled to provide various XR functionalities as described herein. TheXR system 100 and the wearable device 150 can include separate devicesused for XR experiences. In some examples, the XR system 100 canimplement one or more XR applications such as, for example and withoutlimitation, a video game application, a robotic application, anautonomous driving or navigation application, a productivityapplication, a social media application, a communications application, amedia application, an electronic commerce application, and/or any otherXR application.

In some examples, the XR system 100 can include an electronic deviceconfigured to use information about the relative pose of the XR system100 and/or the wearable device 150 to provide one or morefunctionalities, such as XR functionalities (e.g., tracking, detection,classification, mapping, content rendering, etc.), gamingfunctionalities, autonomous driving or navigation functionalities,computer vision functionalities, robotic functions, etc. For example, insome cases, the XR system 100 can be an XR device (e.g., a head-mounteddisplay, a heads-up display device, smart glasses, etc.) and thewearable device 150 can provide tracking measurements to the XR system100 for use by the XR system 100 to aid tracking operations, reducepower usage, support privacy mode operations, etc., as further describedherein.

In the illustrative example shown in FIG. 1 , the XR system 100 caninclude one or more image sensors, such as image sensor 102 and imagesensor 104, other sensors 106, and one or more compute components 110.The other sensors 106 can include, for example and without limitation,an inertial measurement unit (IMU), a radar, a light detection andranging (LIDAR) sensor, an audio sensor, a position sensor, a pressuresensor, a gyroscope, an accelerometer, a microphone, and/or any othersensor. In some examples, the XR system 100 can include additionalsensors and/or components such as, for example, a light-emitting diode(LED) device, a storage device, a cache, a communications interface, adisplay, a memory device, etc. An example architecture and examplehardware components that can be implemented by the XR system 100 arefurther described below with respect to FIG. 6 .

Moreover, in the illustrative example shown in FIG. 1 , the wearabledevice 150 includes an IMU 152, an ultrasonic sensor, a pressure sensor156 (e.g., a barometric air pressure sensor and/or any other pressuresensor), and a touch sensor 158 (or tactile sensor). The sensor devicesshown in FIG. 1 are non-limiting examples provided for explanationpurposes. In other examples, the wearable device 150 can include more orless sensors (of the same and/or different types) than shown in FIG. 1 .Moreover, in some cases, the wearable device 150 can include otherdevices such as, for example, a microphone, a display component (e.g.,light-emitting diode display component), etc.

The components shown in FIG. 1 with respect to the XR system 100 and thewearable device 150 are merely illustrative examples provided forexplanation purposes. In other examples, the XR system 100 and/or thewearable device 150 can include more or less components than those shownin FIG. 1 .

The XR system 100 can be part of, or implemented by, a single computingdevice or multiple computing devices. In some examples, the XR system100 can be part of an electronic device (or devices) such as a camerasystem (e.g., a digital camera, an IP camera, a video camera, a securitycamera, etc.), a telephone system (e.g., a smartphone, a cellulartelephone, a conferencing system, etc.), a laptop or notebook computer,a tablet computer, a set-top box, a smart television, a display device,a gaming console, an XR device such as an HMD, a drone, a computer in avehicle, an IoT (Internet-of-Things) device, a smart wearable device, orany other suitable electronic device(s). In some implementations, theimage sensor 102, the image sensor 104, the one or more other sensors106, and/or the one or more compute components 110 can be part of thesame computing device.

For example, in some cases, the image sensor 102, the image sensor 104,the one or more other sensors 106, and/or the one or more computecomponents 110 can be integrated with or into a camera system, asmartphone, a laptop, a tablet computer, a smart wearable device, an XRdevice such as an HMD, an IoT device, a gaming system, and/or any othercomputing device. However, in other implementations, the image sensor102, the image sensor 104, the one or more other sensors 106, and/or theone or more compute components 110 can be part of, or implemented by,two or more separate computing devices.

The one or more compute components 110 of the XR system 100 can include,for example and without limitation, a central processing unit (CPU) 112,a graphics processing unit (GPU) 114, a digital signal processor (DSP)116, and/or an image signal processor (ISP) 118. In some examples, theXR system 100 can include other types of processors such as, forexample, a computer vision (CV) processor, a neural network processor(NNP), an application-specific integrated circuit (ASIC), afield-programmable gate array (FPGA), etc. The XR system 100 can use theone or more compute components 110 to perform various computingoperations such as, for example, extended reality operations (e.g.,tracking, localization, object detection, classification, poseestimation, mapping, content anchoring, content rendering, etc.),image/video processing, graphics rendering, machine learning, dataprocessing, modeling, calculations, and/or any other operations.

In some cases, the one or more compute components 110 can include otherelectronic circuits or hardware, computer software, firmware, or anycombination thereof, to perform any of the various operations describedherein. In some examples, the one or more compute components 110 caninclude more or less compute components than those shown in FIG. 1 .Moreover, the CPU 112, the GPU 114, the DSP 116, and the ISP 118 aremerely illustrative examples of compute components provided forexplanation purposes.

The image sensor 102 and the image sensor 104 can include any imageand/or video sensor or capturing device, such as a digital camerasensor, a video camera sensor, a smartphone camera sensor, animage/video capture device on an electronic apparatus such as atelevision or computer, a camera, etc. In some cases, the image sensor102 and/or the image sensor 104 can be part of a camera or computingdevice such as a digital camera, a video camera, an IP camera, asmartphone, a smart television, a game system, etc. Moreover, in somecases, the image sensor 102 and/or the image sensor 104 can includemultiple image sensors, such as rear and front sensor devices, and canbe part of a dual-camera or other multi-camera assembly (e.g., includingtwo camera, three cameras, four cameras, or other number of cameras).

In some examples, the image sensor 102 and/or the image sensor 104 cancapture image data and generate frames based on the image data and/orprovide the image data or frames to the one or more compute components110 for processing. A frame can include a video frame of a videosequence or a still image. A frame can include a pixel arrayrepresenting a scene. For example, a frame can be a red-green-blue (RGB)frame having red, green, and blue color components per pixel; a luma,chroma-red, chroma-blue (YCbCr) frame having a luma component and twochroma (color) components (chroma-red and chroma-blue) per pixel; or anyother suitable type of color or monochrome picture.

In some examples, the one or more compute components 110 can perform XRprocessing operations based on data from the image sensor 102, the imagesensor 104, the one or more other sensors 106, and/or the wearabledevice 150. For example, in some cases, the one or more computecomponents 110 can perform tracking, localization, object detection,object classification, pose estimation, shape estimation, mapping,content anchoring, content rendering, image processing, modeling,content generation, gesture detection, gesture recognition, and/or otheroperations based on data from the image sensor 102, the image sensor104, the one or more other sensors 106, and/or the wearable device 150.

In some examples, the one or more compute components 110 can implementone or more algorithms for tracking and estimating a relative pose ofthe wearable device 150 and the XR system 100. In some cases, the one ormore compute components 110 can receive image data captured by the imagesensor 102 and/or the image sensor 104 and perform pose estimation basedon the received image data to calculate a relative pose of the wearabledevice 150 and the XR system 100. Moreover, the one or more computecomponents 110 can receive sensor data (e.g., data from the IMU 152, theultrasonic sensor 154, the pressure sensor 156, and/or the touch sensor158) from the wearable device 150, and use such data to track thewearable device 150 (with or without other data from the image sensor102, the image sensor 104, or the other sensor(s) 106), adjustprocessing/power operations, etc., as described herein. In some cases,the one or more compute components 110 can implement one or morecomputer vision models to calculate the relative pose of the wearabledevice 150 and the XR system 100.

In some cases, the one or more other sensors 106 can detect accelerationby the XR system 100 and generate acceleration measurements based on thedetected acceleration. In some cases, the one or more other sensors 106can additionally or alternatively detect and measure the orientation andangular velocity of the XR system 100. For example, the one or moreother sensors 106 can measure the pitch, roll, and yaw of the XR system100. In some examples, the XR system 100 can use measurements obtainedby the one or more other sensors 106 to calculate the relative pose ofthe XR system 100. In some cases, the XR system 100 can additionally oralternatively use sensor data from the wearable device 150 to performtracking, pose estimation, and/or other operations.

The wearable device 150 can use the IMU 152, the ultrasonic sensor 154,and/or the pressure sensor 156 to obtain tracking measurements for thewearable device 150. The tracking measurements can include, for exampleand without limitation, position measurements, velocity/motionmeasurements, range/distance measurements, elevation measurements, etc.The wearable device 150 can provide the tracking measurements to the XRsystem 100. In some cases, the wearable device 150 can use the touchsensor 158 to receive user inputs for the XR system 100, such as forexample, touch inputs (e.g., tapping, squeezing, pressing, rubbing,touching, etc.). The wearable device 150 can provide one or moredetected inputs to the XR system 100 to modify a content, operation,and/or behavior of the XR system 100.

In some examples, the XR system 100 can use measurements obtained by theIMU 152, the ultrasonic sensor 154, and/or the pressure sensor 156 tocalculate (and/or to assist in calculating) the relative location,motion, and/or position of the wearable device 150. In some cases, theIMU 152 can detect acceleration by the wearable device 150 and generateacceleration measurements based on the detected acceleration. In somecases, the IMU 152 can additionally or alternatively detect and measurethe orientation and angular velocity of the wearable device 150. Forexample, the IMU 152 can measure the pitch, roll, and yaw of thewearable device 150.

In some examples, the wearable device 150 can use the ultrasonic sensor154 for ranging. For example, the ultrasonic sensor 154 can measure adistance of the wearable device 150 relative to the XR system 100 and/oranother object such as, for example, another wearable device, a bodypart (e.g., a hand, etc.), a person, a wall, another device (e.g., acontroller, a stylus, a joystick, a mouse, a display, etc.), and/or anyother object. The pressure sensor 156 can detect pressure such as airpressure, and can determine relative elevation changes. The touch sensor158 can measure physical forces or interactions with the wearable device150, which can be interpreted as inputs to the XR system 100.

The wearable device 150 can include one or more wireless communicationinterfaces (not shown) for communicating with the XR system 100. In someexamples, the one or more wireless communication interfaces can includea wireless transmitter, a wireless transceiver, or any other means forwireless communications and/or transmitting data. The one or morewireless communication interfaces can implement any wireless protocoland/or technology to communicate with the XR system 100, such asshort-range wireless technologies (e.g., Bluetooth, etc.) for example.The wearable device 150 can use the one or more wireless communicationinterfaces to transmit sensor measurements and/or other XR inputs to theXR system 100, as further described herein.

While the XR system 100 and the wearable device 150 are shown to includecertain components, one of ordinary skill will appreciate that the XRsystem 100 and the wearable device 150 can include more or fewercomponents than those shown in FIG. 1 . For example, the XR system 100and/or the wearable device 150 can also include, in some instances, oneor more other memory devices (e.g., RAM, ROM, cache, and/or the like),one or more networking interfaces (e.g., wired and/or wirelesscommunications interfaces and the like), one or more display devices,caches, storage devices, and/or other hardware or processing devicesthat are not shown in FIG. 1 . An illustrative example of a computingdevice and/or hardware components that can be implemented with the XRsystem 100 and/or the wearable device 150 described below with respectto FIG. 6 .

FIG. 2A illustrates an example of a ring device 200 worn on a finger 210of a user interacting with the XR system 100. The ring device 200 can bean example of a wearable device, such as wearable device 150 shown inFIG. 1 . For example, in some cases, the ring device 200 can be the sameas the wearable device 150 shown in FIG. 1 . In other cases, the ringdevice 200 can be a wearable device with a different form factor thanthe wearable device 150 shown in FIG. 1 and/or a different type ofwearable device than the wearable device 150 shown in FIG. 1 .

The user can use the ring device 200 to provide tracking measurements tothe XR system 100, such as position measurements, distance measurements,elevation measurements, motion measurements, location measurements, etc.In some cases, the user can use the ring device 200 to interact with theXR system 100 and provide XR inputs. In some examples, the ring device200 can collect sensor measurements to track a location, motion, pose,etc., of the ring device 200 in 3D space. In some examples, thelocation, motion, pose, etc., can be tracked relative to a location,motion, pose, etc., of the XR system 100 in 3D space. In some cases, thelocation, motion, pose, etc., can be tracked relative to a location,motion, pose, etc., of another object in 3D space, such as anotherwearable device, a person, a hand, a leg, a wall, an input device (e.g.,a stylus, a controller, a mouse, etc.), an animal, a vehicle, etc.

In the example shown in FIG. 2A, the ring device 200 includes a touchpad204 for receiving touch inputs, a display 206 for displaying informationfrom the ring device 200 and/or the XR system 100, and sensors 208.However, in other cases, the ring device 200 can include more or lesssensors/devices than shown in FIG. 2A. In some examples, the sensors 208can include and/or can be the same as the IMU 152, the ultrasonic sensor154, the pressure sensor 156, and/or the touch sensor 158 shown in FIG.1 . In other examples, the sensors 208 can include one or more sensorsthat are not shown in FIG. 1 . In some cases, the ring device 200 caninclude a touch or pressure sensitive surface and/or surface portion formeasuring touch inputs.

The XR system 100 can render content, interfaces, controls, etc., to auser wearing the XR system 100 and the ring device 200. The user can usethe ring device 200 to wirelessly provide tracking measurements to theXR system 100 to assist the XR system 100 in tracking, detecting,classifying, mapping, etc., one or more objects associated with the ringdevice 200, such as the finger 210, a hand of the finger 210, the ringdevice 200, etc. In some examples, the user can use the ring device 200to wirelessly adjust operations of the XR system 100; interact with XRcontent/interfaces/controls/etc., presented at the XR system 100; and/orprovide XR inputs such as selections, object/environment manipulations,navigation inputs (e.g., scrolling, moving, etc.), gestures, etc.

The ring device 200 can be used to provide data to the XR system 100 toenhance tracking at the XR system 100, reduce power consumption at theXR system 100 when providing an XR experience, trigger a privacy mode(and a termination of a privacy mode) at the XR system 100, enhance XRfunctionalities, etc., as further described herein. For example, withreference to FIG. 2B, at time T₁, the ring device 200 can obtain data220 based at least partly based on sensor data from one or more sensors(e.g., touchpad 204, sensors 208, etc.) on the ring device 200, and sendthe data 220 to the XR system 100. In some examples, the data 220 caninclude tracking measurements such as, for example, positionmeasurements, distance measurements, elevation measurements, motionmeasurements, location measurements, etc. In some examples, the data 220can include pose information associated with the ring device 200. Forexample, the data 220 can include a pose of the ring device 200 inphysical space, a pose of the ring device 200 relative to the XR system100, a pose of the ring device 200 in a coordinate system of the ringdevice 200, and/or a pose of the ring device 200 in a coordinate systemof the XR system 100. In some examples, the data 220 can include commandand/or input information. For example, in some cases, the data 220 caninclude one or more commands and/or inputs for triggering or stopping anoperation at the XR system 100, triggering a setting and/or operationstate/mode of the XR system 100, providing an input to an application onthe XR system 100, and/or any other commands and/or inputs.

In some examples, the XR system 100 can use the data 220 to track thering device 200 at T₁. In some cases, the XR system 100 can use the data220 to estimate a position of the ring device 200 at one or more timesteps after T₁. In some examples, the XR system 100 can use the data 220to determine whether the finger 210 (and/or a hand of the finger 210) isvisible to one or more image sensors (e.g., image sensor 102, imagesensor 104) on the XR system 100. For example, the XR system 100 can usethe data 220 to determine whether the finger 210 (and/or a hand of thefinger 210) is within a FOV of one or more image sensors (e.g., imagesensor 102, image sensor 104) on the XR system 100 and/or whether a viewof the one or more image sensors to the finger 210 (and/or the hand ofthe finger 210) is obstructed by one or more objects. The XR system 100can use this information to adjust device (e.g., image sensor) settings,XR operations, etc., as further described herein.

At time T_(n), the ring device 200 can obtain data 230 based at leastpartly based on sensor data from the one or more sensors on the ringdevice 200, and send the data 230 to the XR system 100. The XR system100 can use the data 230 to track the ring device 200 at T_(n). In somecases, the XR system 100 can use the data 230 to estimate a position ofthe ring device 200 at one or more time steps after T_(n). In someexamples, the XR system 100 can use the data 230 to determine whetherthe finger 210 (and/or a hand of the finger 210) is visible to one ormore image sensors on the XR system 100, as previously explained.

As previously mentioned, XR systems may track objects (e.g., hands,fingers, devices, etc.) based on image data received by cameras on theXR systems. However, persistently or periodically capturing and/orprocessing images can be costly. Moreover, in some cases, capturingand/or processing images can result in unnecessary usage of powerresources. For example, an object of interest being tracked by an XRsystem may be occluded from a view of the camera(s) on the XR system,preventing the XR system from capturing the object of interest in animage obtained from the camera(s) on the XR system. As another example,there may be periods of time where a user may not intend to provide anyinput to the XR system 100 using the object of interest (or at all),there may be spaces where camera imaging is not permitted or thelighting conditions are poor, etc.

In some examples, data from the wearable device 150 can be used to helpthe XR system 100 in tracking an object of interest, save power at theXR system 100, control one or more operations/modes at the XR system100, etc. For example, the wearable device 150 can receive signals fromthe wearable device 150 to inform a tracking engine in the XR system100. In some cases, the signals can include data from and/or data basedon sensor data from sensors embedded in the wearable device 150. The XRsystem 100 can use the signals from the wearable device 150 to track abody part (e.g., a finger, hand, wrist, group of fingers, etc.) wearingthe wearable device 150 and correlate an estimated location of the bodypart with a FOV of any image sensors (e.g., image sensor 102, imagesensor 104) on the XR system 100. In some examples, the XR system 100can use this information to limit imaging to those image sensor(s) thatcan see the body part at a given time. In some cases, when the body partis not observable by an image sensor (e.g., the body part is outside ofthe FOV of the image sensor, the body part is occluded by one or moreobjects, the lighting conditions in the environment are poor or toodark, etc.), the XR system 100 can maintain tracking of the body partbased on sensor signals from the wearable device 150 (e.g., positionmeasurements, motion measurements, a pressure difference between the XRsystem 100 and the wearable device 150, location measurements, distancemeasurements, etc.).

In some cases, the XR system 100 can use the signals from the wearabledevice 150 to inform a visual object tracking engine at the XR system100 when an object of interest commences a movement towards anobservable FOV associated with one or more image sensors on the XRsystem 100. For example, the XR system 100 can use signals indicating avelocity of a hand moving into view to pre-seed the hand tracking.Moreover, the wearable device 150 and associated signals can help savepower on the XR system 100 by limiting a use of one or more imagesensors for tracking to instances where the object of interest isobservable by the one or more image sensors. In some examples, a userexperience can be improved by providing for observable gestures with anobject (e.g., a hand, etc.) and/or gestures with an object (e.g., ahand, etc.) that is not in a FOV of the image sensor(s) on the XR system100. The XR system 100 can improve a tracking robustness by combiningsignals from the wearable device 150 with image data from one or moreimage sensors on the XR system 100. In some cases, the signals from thewearable device 150 can be also be used to switch an operation/mode ofthe XR system 100 to a non-imaging mode (e.g., a private mode) in aprivate location (e.g., a restroom, locker room, an office, etc.) and/orduring periods when the user wishes to maintain a privacy by preventingimages from being captured.

FIG. 3 is a diagram illustrating an example process 300 for integratingdata from a wearable device 150 for XR operations at an XR system 100.The blocks, steps, and/or operations outlined in the example process 300are illustrative examples and can be implemented in a different orderand/or in any combination, including combinations that exclude, add, ormodify certain blocks, steps, and/or operations.

In this example, the wearable device 150 can send data 302 to the XRsystem 100. The data 302 can include data calculated, measured, and/orcollected by one or more sensors on the wearable device 150, such as IMU152, ultrasonic sensor 154, pressure sensor 156, and/or touch sensor158. In some cases, the data 302 can additionally or alternativelyinclude can include data generated at least partly based on datacalculated, measured, and/or collected by one or more sensors on thewearable device 150. In some cases, the wearable device 150 can obtainthe data 302 while the wearable device 150 is worn by a user. Forexample, the wearable device 150 can obtain and send the data 302 whilea user wears the wearable device 150 on a finger, wrist, or other bodypart. The data 302 can include, for example and without limitation, oneor more position measurements, distance measurements, elevationmeasurements, location measurements, motion measurements, commands,inputs, pose information, etc.

At block 304, the XR system 100 can use the data 302 to determine camera(e.g., image sensor 102, image sensor 104) operating settings. The XRsystem 100 can adjust the camera operating settings to balance a powerconsumption of the image sensor(s) at the XR system 100 with the abilityof the image sensor(s) to capture an image(s) (or a thresholdquality/resolution image) of the wearable device 150 and/or a targetassociated with the wearable device 150 such as a hand, a finger, etc.For example, the XR system 100 can adjust camera operating settings toreduce power consumption by image sensors at the XR system 100 when suchimage sensors are unable to capture an image of the wearable device 150(or an image with a threshold quality) and/or the target associated withthe wearable device 150, and increase or maintain a power consumption bythe image sensors when the image sensors are able to capture an image ofthe wearable device 150 and/or the target associated with the wearabledevice 150.

The camera operating settings can include, for example, a power state(e.g., turned off, turned on, a higher power state, a lower/reducedpower state, etc.) of one or more image sensors at the XR system 100, asetting (e.g., a framerate, a resolution, etc.) of one or more imagesensors, an active/inactive state of one or more image sensors, XRoperations enabled/implemented by one or more image sensors, camerahardware used by one or more image sensors, and/or any other state orconfiguration of one or more image sensors and/or associated hardware.In some examples, the XR system 100 can determine the camera operatingsettings based on a position, motion, location, and/or state of thewearable device 150, as determined based on the data 302.

For example, if the XR system 100 determines (e.g., at least partlybased on the data 302) that the wearable device 150 is outside of a FOVof one or more image sensors on the XR system 100 or a view of the oneor more image sensors to the wearable device 150 is obstructed (e.g., byan object, poor lighting conditions, background cluttering, etc.), theXR system 100 can turn off/disable the one or more image sensors, reducea framerate of the one or more image sensors, reduce a resolution of theone or more image sensors, reduce operations of the one or more imagesensors, and/or otherwise reduce a power consumption or state of the oneor more image sensors, to avoid unnecessary use of and/or powerconsumption by the one or more image sensors while the one or more imagesensors are unable to capture an image of the wearable device 150 (or animage from which the wearable device 150 can be detected).

As another example, if the XR system 100 determines (e.g., at leastpartly based on the data 302) that the wearable device 150 is visible toone or more image sensors on the XR system 100 (e.g., based on a FOV ofthe one or more image sensors, lighting conditions, etc.), the XR system100 can turn on/enable or maintain on the one or more image sensors,increase or maintain a certain framerate of the one or more imagesensors, increase or maintain a resolution of the one or more imagesensors, increase or maintain certain operations of the one or moreimage sensors, and/or otherwise increase or maintain a power consumptionor state of the one or more image sensors. This can allow the XR system100 to capture an image(s), a higher quality image(s), a higher rate ofconsecutive images, etc., for use in one or more XR operations such as,for example, tracking, object detection, classification, mapping, etc.

In some cases, if all the image sensors at the XR system 100 are turnedoff/disabled (and/or the wearable device 150 is not visible to any ofthe image sensors at the XR system 100), the XR system 100 can use thedata 302 to track the wearable device 150 and/or a target associatedwith the wearable device 150, until the wearable device 150 and/or thetarget is/are visible to one or more image sensors at the XR system 100.In some cases, the XR system 100 can turn on or enable an image sensorwhen the wearable device 150 and/or the target becomes visible to theimage sensor (e.g., as determined based on sensor data from the wearabledevice 150) or approaches (or is within) a threshold distance from a FOVof the image sensor (e.g., as determined based on sensor data from thewearable device 150).

In some cases, the XR system 100 can use the sensor data from thewearable device 150 to determine a trajectory of the wearable device 150(and/or the target associated with the wearable device 150) relative tothe FOV of one or more image sensors on the XR system 100. In someexamples, the XR system 100 can use this information to (increasingly)reduce or increase an operating state and/or setting of an image sensoras the wearable device 150 moves closer or farther from a FOV of theimage sensor. For example, the XR system 100 can increasingly reduce aframerate, resolution, power state, etc., of an image sensor as thewearable device 150 moves farther from the FOV of the image sensor.Similarly, the XR system 100 can increase the framerate, resolution,power state, etc., of the image sensor as the wearable device 150 movescloser to the FOV of the image sensor.

In some cases, if an image sensor at the XR system 100 is turnedon/enabled and the wearable device 150 (and/or a target associated withthe wearable device 150) is visible to the such image sensor, the XRsystem 100 can use the image sensor to capture one or more images of thewearable device 150 (and/or a target associated with the wearable device150), which the XR system 100 can use to track the wearable device 150(and/or a target associated with the wearable device 150). The XR system100 can also use the data 302 to aid in the tracking of the wearabledevice 150 (and/or a target associated with the wearable device 150), asthe data 302 can provide additional and/or corroborating informationabout a position, location, motion, etc., of the wearable device 150,and/or can help increase a tracking fidelity/accuracy.

In some examples, depending on the view to the wearable device 150 ofeach image sensor at the XR system 100 (e.g., as determined at leastpartly based on the data 302), the XR system 100 can configure differentimage sensors in different operating modes. For example, if the wearabledevice 150 is within a FOV of an image sensor and outside of the FOV ofa different image sensor, the XR system 100 can turn off/disable orreduce a setting of (e.g., a framerate, a resolution, etc.) thedifferent image sensor that does not have a view to the wearable device150. The XR system 100 can also maintain on/enabled and/or increase asetting of (e.g., a framerate, a resolution, etc.) of the image sensorthat does have a view to the wearable device 150.

At block 306, the XR system 100 can use the data 302 to determine XRprocessing settings implemented by the XR system 100. The XR system 100can adjust XR processing settings to balance power consumption at the XRsystem 100 with a potential performance (e.g., accuracy, etc.) and/orbenefit of the XR processing settings. For example, the XR system 100can adjust certain XR processing settings to reduce power consumption ifthe XR system 100 determines that a reduction in power consumptionresulting from such adjustment outweighs a potential (if any) negativeperformance impact, and vice versa.

In some examples, the XR processing settings can define a trackingworkflow and/or a frequency of one or more operations in a trackingworkflow. For example, the XR processing settings can define whichtracking, object detection, classification, etc., operations to beperformed and/or a frequency of such operations. To illustrate, the data302 may indicate that a hand (or a finger of a hand) wearing thewearable device 150 is outside of the FOV of the image sensor(s) of theXR system 100. The XR system 100 can thus stop certain trackingoperations, such as hand detection and/or hand classificationoperations, while the wearable device 150 and the hand are outside ofthe FOV of the image sensor(s).

In some cases, the XR system 100 can save power by reducing the numberand/or frequency of tracking operations while the wearable device 150and the hand (or any other target) are outside of the FOV of the imagesensor(s). For example, when the wearable device 150 and the hand areoutside of the FOV of the image sensor(s), rather than performing handdetection and/or hand classification operations on every image capturedby the image sensor(s), the XR system 100 can perform the hand detectionand/or hand classification operations every n number of images capturedby the image sensor(s).

At block 308, the XR system 100 can detect a target (e.g., the wearabledevice 150, a hand associated with the wearable device 150, a fingerassociated with the wearable device 150, etc.) in an image(s) capturedby one or more image sensors of the XR system 100. The one or more imagesensors of the XR system 100 can capture the image when the target iswithin a FOV of the one or more image sensors as determined at leastpartly based on the data 302. The XR system 100 can implement an objectdetection algorithm to detect the target in the image. In some examples,the XR system 100 can implement computer vision and/or machine learningto detect the target in the image. In some cases, the XR system 100 canuse the data 302 and/or image data from one or more image sensors on theXR system 100 to detect and/or recognize a gesture of the target, modifycontent (e.g., virtual content, interfaces, controls, etc.) rendered bythe XR system 100, generate inputs/interactions with content rendered bythe XR system 100, etc.

In some cases, the XR system 100 can use the data 302 to reduce anamount of power used to detect the target in the image. For example, insome cases, the XR system 100 can use the data 302 to predict where inthe image to search for the target. The XR system 100 can reduce thepower consumed by the XR system 100 in searching for the target in theimage by reducing the search time and/or search field. For example,using the data 302, the XR system 100 can search for the target in oneor more regions of the image without searching the entire image. Thedata 302 can provide an indication of the location/position of thetarget, which the XR system 100 can use to determine which region(s) ofthe image to search. By reducing the search, the XR system 100 canreduce the power consumed by the tracking algorithm (and the efficiencyof the tracking algorithm) in finding the target within the image. TheXR system 100 can thus avoid searching unnecessary regions of the image.

At block 310, the XR system 100 can estimate a pose and/or shape of thetarget detected in the image(s). In some examples, the XR system 100 canimplement a machine learning algorithm to estimate the pose and/or shapeof the target detected in the image(s). For example, the XR system 100can implement one or more neural networks to process the data 302 and/orthe image(s) and determine a shape and/or pose of the target. In somecases, the XR system 100 can use the estimated pose and/or shape of thetarget for one or more XR operations such as, for example,tracking/localization, mapping, content anchoring, content rendering,etc.

FIG. 4 is a flow diagram illustrating an example process 400 for using awearable device 150 with an XR system 100 for enhanced tracking, powersavings, and privacy functionalities. The blocks/operations outlined inthe example process 400 are illustrative examples and can be implementedin a different order and/or in any combination, including combinationsthat exclude, add, or modify certain blocks/operations.

In this example, at block 402, the XR system 100 can capture one or moreimages of a target associated with the wearable device 150. In somecases, the target can include, for example, a body part wearing thewearable device 150 or associated with another body part wearing thewearable device 150, such as a finger wearing the wearable device, ahand of the finger wearing the wearable device 150 (or wearing thewearable device 150), etc. In some cases, the target can include anobject such as, for example, the wearable device 150, an input device(e.g., a controller, a stylus, a mouse, etc.), a person, an animal, aseparate device (e.g., a robotic device, etc.), a vehicle, or anyobject.

At block 404, the wearable device 150 can send data to the XR system100. The data can include data captured/measured by one or more sensorsat the wearable device 150 such as, for example, position measurements,motion measurements, distance measurements, location measurements,elevation measurements, etc. In some cases, the data can additionally oralternatively include data generated at least partly based on datacaptured/measured by one or more sensors at the wearable device 150. Insome examples, the data can include position information. For example,the data can include a pose of the wearable device 150. The pose can be,for example, a pose in physical space (e.g., in 3D space), a poserelative to the XR system 100, a pose within a coordinate system of thewearable device 150, a pose within a coordinate system of the XR system100, any combination thereof, and/or any other pose. In some examples,the data can include one or more commands and/or inputs to the XR system100, such as an application command/input, a command/input to apply oneor more operating states/modes, one or more settings, one or moreoperations, etc.

At block 406, the XR system 100 can use the one or more images of thetarget and the data from the wearable device 150 to track the target.For example, the XR system 100 can detect a location of the targetwithin the one or more images and use the location of the target withinthe one or more images to estimate a position of the target in 3D space.The XR system 100 can use the data to help detect the target within theone or more images and/or determine the position of the target within 3Dspace. In some examples, the XR system 100 can implement a computervision algorithm and/or a machine learning (e.g., a neural network,etc.) algorithm to process the one or more images and the data to trackthe target.

At block 408, the wearable device 150 can send additional data to the XRsystem 100. At block 410, the XR system 100 can use the data todetermine a visibility of one or more image sensors at the XR system 100to the target. For example, the XR system 100 can use the data from thewearable device 150 to determine the position of the wearable device 150and the target in 3D space. Based on the position of the target in 3Dspace, the XR system 100 can determine whether the target is within theFOV of any image sensor on the XR system 100. The XR system 100 can thusdetermine whether the target is outside or inside the FOV of any imagesensor on the XR system 100.

At block 412, the XR system 100 can adjust operating and/or processingsettings at the XR system 100 based on the position of the targetrelative to the FOV of each image sensor on the XR system 100. Theoperating and/or processing settings can include, for example andwithout limitation, camera device settings (e.g., framerate, resolution,power mode, resource use, state, etc.), XR processing or workflowsettings, etc. In some examples, the XR processing or workflow settingscan define specific XR operations to be implemented by the XR system 100(e.g., object detection, object classification, pose estimation, shapeestimation, mapping, gesture detection, gesture recognition, etc.), afrequency in which any particular XR operation is (or is not)implemented (e.g., based on units of time, number of frames processed,events, etc.), a setting of any particular XR operation implemented(e.g., a full implementation, a partial implementation, a coarseimplementation, a sparse implementation, a fidelity, etc.), and/or anyother processing setting.

For example, the XR system 100 can adjust an image sensor's settings toenable a use of the image sensor (e.g., turn on, enable, etc.) and/orincrease a performance of the image sensor (e.g., increase a framerate,resolution, power mode, resource use, etc.) if the target is within theFOV (or nearing the FOV within a threshold proximity and/or estimatedtimeframe) of the image sensor, or disable a use of the image sensor(e.g., turn off, disable) and/or decrease a performance and/or powerconsumption of the image sensor (e.g., decrease a framerate, resolution,power mode, resource use, etc.) if the target is not within the FOV (ornot nearing the FOV within a threshold proximity and/or estimatedtimeframe) of the image sensor. In some cases, the XR system 100 canimplement different image sensor settings for some or all of the imagesensors on the XR system 100. In other cases, the XR system 100 canimplement the same image sensor settings for every image sensor on theXR system 100.

As another example, the XR system 100 can (additionally oralternatively) enable (or increase a performance/processing setting of)one or more operations such as object detection, object classification,pose estimation, shape estimation, etc., if the target is within the FOV(or nearing the FOV within a threshold proximity and/or estimatedtimeframe) of the image sensor, or disable (or decrease aperformance/processing setting of) one or more operations such as objectdetection, object classification, pose estimation, shape estimation,etc., if the target is not within the FOV (or not nearing the FOV withina threshold proximity and/or estimated timeframe) of the image sensor.

At block 414, the wearable device 150 can send additional data to the XRsystem 100. At block 416, the XR system 100 can use the data from thewearable device 150 to track the target. If the target is not within theFOV of any image sensor on the XR system 100 and/or if the XR system 100does not have an image capturing the wearable device 150 and/or targetat a position associated with the data (or if the XR system 100 isunable to detect the target from any captured image), the XR system 100can rely on the data from the wearable device 150 to track the target.

For example, if at block 412 the XR system 100 turns off or disablesevery image sensor at the XR system 100 (e.g., because the target is notwithin the FOV of any image sensor or not nearing the FOV within athreshold proximity and/or estimated timeframe), the XR system 100 maynot have an image of the target at a current location of the target (orcaptured within a threshold time period). Accordingly, without an imageof the target that the XR system 100 can use to track the target, the XRsystem 100 can rely on the data from the wearable device 150 to trackthe target until the XR system 100 is able to obtain an image of thetarget (or tracking is terminated).

On the other hand, if at block 412 the XR system 100 does not turn offor disable every image sensor at the XR system 100 and at least oneimage sensor is able to capture an image of the target, the XR system100 can use the image of the target as well as the data from thewearable device 150 to track the target. In some examples, the XR system100 can use the data along with the image of the target to determine aposition of the target in 3D space. In some cases, the XR system 100 canuse the data to help find the target in the image. For example, the XRsystem 100 can use the data to estimate what region(s) of the imagecontains the target and/or reduce what regions of the image it searchesfor the target. In some cases, by reducing the regions of the imagesearched (e.g., instead of searching the full image), the XR system 100can reduce an amount of power consumed in searching the target in theimage and/or increase an efficiency of finding the target in the image.

At block 418, the wearable device 150 can send additional data to the XRsystem 100. At block 420, the XR system 100 can enable or increase oneor more operating/processing settings at the XR system 100 (e.g., imagesensor settings, XR processing or workflow settings, etc.). For example,the data can trigger the XR system 100 to enable or increase one or moreoperating/processing settings previously disabled or decreased (e.g., atblock 412). The XR system 100 can use the data from the wearable device150 to determine a position of the target and determine whether toenable or increase any operating/processing settings based on theposition of the target relative to the FOV of one or more image sensorson the XR system 100. In some examples, the XR system 100 can use thedata to determine a visibility of each image sensor on the XR system 100to the target, and enable or increase any operating/processing settingsbased on the visibility of each image sensor to the target.

For example, if the XR system 100 previously (e.g., at block 412) turnedoff or disabled an image sensor and subsequently determines (e.g., atblock 420) that the target is within the FOV (or nearing the FOV withina threshold proximity and/or estimated timeframe) of the image sensor,the XR system 100 can turn on or enable that image sensor in order tocapture an image(s) of the target using that image sensor. The XR system100 can use such image of the target to track the target, detect thetarget, estimate a pose of the target, estimate a shape of the target,etc., as described herein. In some examples, if the XR system 100previously (e.g., at block 412) turned off or disabled an image sensorand determines (e.g., at block 420) that the target is not (or is stillnot) within the FOV (or not nearing the FOV within a threshold proximityand/or estimated timeframe) of the image sensor, the XR system 100 canmaintain that image sensor in the off or disabled state to conservepower.

At block 422, the XR system 100 can use one or more image sensors thatare turned on/enabled to capture an image(s) of the target. The one ormore image sensors can include any image sensor on the XR system 100having visibility to the target. At block 424, the wearable device 150can also send data to the XR system 100.

At block 426, the XR system 100 can use the image(s) of the target andthe data from the wearable device 150 to track the target, as previouslydescribed.

At block 428, the wearable device 150 can send additional data to the XRsystem 100. At block 430, the data from the wearable device 150 cantrigger a privacy mode at the XR system 100. The privacy mode caninclude turning off or disabling each image sensor on the XR system 100.

In some examples, the data that triggers the privacy mode can indicate acertain input (e.g., a touch input, a motion-based input, etc.) providedto the wearable device 150 and/or a certain state of the wearable device150 (e.g., the wearable device 150 was removed from a finger, hand, orother body part; the wearable device 150 was placed on a surface; etc.).The XR system 100 can interpret the particular input and/or stateindicated in the data as a request or an intent to enter/enable theprivacy mode.

In some examples, the wearable device 150 can send additional data tothe XR system 100 to trigger the XR system 100 to stop the privacy mode.For example, the wearable device 150 can send additional data indicatinga certain input (e.g., a touch input, a motion-based input, etc.) and/ora certain state of the wearable device 150 (e.g., the wearable device150 was placed on a finger, hand, or other body part), which the XRsystem 100 can interpret as a request or intent to stop the privacymode. The XR system 100 can stop the privacy mode and return to previousoperating/processing settings or use data from the wearable device 150to determine operating/processing settings to implement after theprivacy mode.

In some cases, when the XR system 100 is in privacy mode, the XR system100 can use data provided by the wearable device 100 to provide somelevel of tracking of the target. For example, the wearable device 150can send data to the XR system 100 while the XR system 100 is in privacymode. The XR system 100 can use the data to track the target aspreviously explained.

FIG. 5A is a flowchart illustrating an example process 500 for using awearable device (e.g., wearable device 150, wearable ring 200) with anXR device (e.g., XR system 100). In some examples, the process 500 canuse the wearable device with the XR device to enhance tracking, powersavings, and/or privacy functionalities at the XR device.

At block 502, the process 500 can include establishing a wirelessconnection between the wearable device and the XR device. The wirelessconnection can be established using a wireless communication interface(e.g., a wireless transmitter, wireless transceiver, or any other meansfor transmitting data) on the wearable device and a wirelesscommunication interface on the XR device. The wireless communicationinterface on the wearable device can implement any wireless protocoland/or technology to communicate with the XR device, such as short-rangewireless technologies (e.g., Bluetooth, etc.) for example. In somecases, the wearable device can be paired with the XR device forcommunications/interactions between the wearable device and the XRdevice.

In some examples, the body part associated with the user can include ahand, a finger, multiple fingers, and/or a wrist. In some cases, thewearable device can include a ring. In other cases, the wearable devicecan include a bracelet, a glove, a ring sleeve, or any other wearableitem.

At block 504, the process 500 can include obtaining one or more trackingmeasurements at the wearable device. The wearable device can obtain theone or more tracking measurements using one or more sensors (e.g., IMU152, ultrasonic sensor 154, pressure sensor 156, touch sensor 158) onthe wearable device. In some examples, the one or more sensors caninclude an accelerometer, a gyroscope, a pressure sensor, an audiosensor, a touch sensor, and/or a magnetometer.

In some examples, the one or more tracking measurements can include aposition of the structure in a physical space (e.g., in 3D space),movement of the structure, a distance of the structure relative to oneor more objects, a velocity of the movement of the structure, anelevation of the structure in the physical space, and/or a pose of thestructure in the physical space. In some examples, the one or moreobjects can include the XR device, a different body part (e.g., adifferent finger, a different hand, a different wrist, a different setof fingers, etc.) associated with the user, an input device (e.g., acontroller, a stylus, a mouse, etc.), a wall, a person, an animal, aseparate device, and/or any other object.

In some examples, at least one of the one or more tracking measurementscan be relative to a reference coordinate system of the wearable device(and/or one or more sensors of the wearable device).

At block 506, the process 500 can include sending data associated withthe one or more tracking measurements to the XR device. In some cases,the data can include the one or more tracking measurements. In somecases, the data can additionally or alternatively include data generatedbased at least partly on the one or more tracking measurements. In someexamples, the data can include position information (e.g., a pose of thewearable device), one or more commands, one or more inputs, and/or anyother data and/or sensor measurements. The XR device can use the data totrack a target (e.g., the body part, the wearable device, etc.), adjustone or more device settings and/or operations, trigger a privacy mode,etc. In some cases, the XR device can use the data tovalidate/corroborate tracking results obtained by the XR device based onimage data captured by the XR device. In some cases, the XR device canuse the data to maintain tracking of the body part when the body partcannot be imaged by any image sensors on the XR device (e.g., becausethe body part is outside of a FOV of every image sensor, the XR deviceis operating in a privacy or non-imaging mode, the lighting conditionsin the environment are poor or too dark, the body part is occluded froma view of every image sensor on the XR device, etc.).

In some cases, the XR device can use the data to distinguish the bodypart from other objects (e.g., other body parts, devices, people, etc.)to allow the XR device to detect and track the body part (or the correctbody part). In some cases, the wearable device can send one or moresignals to the XR device with a certain frequency pattern. The frequencypattern can be associated with the wearable device and can be used bythe XR device to recognize the wearable device and the body partassociated with the wearable device. In some examples, the XR device canuse the frequency pattern to distinguish the wearable device and thebody part from other objects in an environment. In some cases, thewearable device can include one or more visual patterns that the XRdevice can detect from one or more images to recognize the wearabledevice in the environment.

In some aspects, the process 500 can include sending, by the wearabledevice (e.g., via the wireless communication interface) to the XRdevice, an input configured to trigger a privacy mode at the XR device.In some examples, the privacy mode can include an operating state withone or more image sensors (e.g., image sensor 102, image sensor 104) atthe XR device being turned off and/or disabled. The input can be basedon the data obtained at block 504 and/or one or more associatedmeasurements from one or more sensors on the wearable device. In someexamples, the data can indicate a touch signal corresponding to a touchinput at the wearable device, a location of the wearable device thatdiffers from a location of the body part, and/or a distance between thewearable device and the body part. For example, the data can include anindication and/or one or more measurements indicating that the wearabledevice is not being worn by the user on the body part (e.g., is at adifferent location and/or within a threshold distance).

In some aspects, the process 500 can include sending, by the wearabledevice (e.g., via the wireless communication interface) to the XRdevice, an additional input configured to trigger the XR device to stopthe privacy mode. The input can be based on data indicating a touchsignal corresponding to a touch input at the wearable device, a locationof the wearable device corresponding to a location of the body part(e.g., indicating that the user may be wearing the wearable device onthe body part), and/or a proximity between the wearable device and thebody part (e.g., indicating that the user may be wearing the wearabledevice on the body part).

In some aspects, the processing 500 can include sending, by the wearabledevice (e.g., via the wireless communication interface) to the XRdevice, an XR input associated with an XR application at the XR device.The XR input can be based on one or more measurements from one or moresensors on the wearable device.

In some aspects, the process 500 can include sending, by the wearabledevice (e.g., via the wireless communication interface) to the XRdevice, an input configured to trigger an adjustment of a device settingat the XR device and/or one or more XR operations at the XR device. Insome examples, the device setting can include a power mode (e.g., off,on, lower power, higher power, etc.) associated with one or more imagesensors at the XR device, a framerate associated with the one or moreimage sensors, a resolution associated with the one or more imagesensors, etc. In some examples, the one or more XR operations caninclude object detection, object classification, gesture detectionand/or recognition, pose estimation, shape estimation, etc.

FIG. 5B is a flowchart illustrating an example process 520 for trackingobjects. In some examples, the process 520 can use a wearable device(e.g., wearable device 150, wearable ring 200) with an electronic device(e.g., XR system 100, etc.) to enhance tracking, power savings, and/orprivacy functionalities at the electronic device. In some examples, theprocess 520 can be implemented by an electronic device, such as XRsystem 100. In some examples, the electronic device can include a mobilephone, a laptop, a tablet, a head-mounted display, smart glasses, acamera system, and/or any other electronic device.

At block 522, the process 520 can include determining a first positionof a wearable device (e.g., wearable device 150, wearable ring 200) in aphysical space. In some examples, determining the first position of thewearable device can include receiving, from the wearable device, imagedata from one or more image sensors on the electronic device and/or dataassociated with one or more measurements from one or more sensors on thewearable device; and determining the first position of the wearabledevice based on the image data from the one or more image sensors and/ordata associated with the one or more measurements from the one or moresensors.

In some examples, the data can include a distance of the wearable devicerelative to one or more objects (e.g., a wall, a door, furniture, adevice, a person, an animal, etc.), a velocity vector indicating avelocity of movement of the wearable device, a touch signal measured bya touch sensor from the one or more sensors, audio data from an audiosensor from the one or more sensors, and/or an elevation of the wearabledevice in the physical space. In some cases, the one or more objects caninclude the electronic device, a body part (e.g., a hand, a leg, an arm,a head, a torso, etc.) associated with a user of the wearable device,and/or an input device (e.g., a controller, a keyboard, a remote, etc.).

In some cases, the wearable device can include a bracelet, a ring, or aglove.

At block 524, the process 520 can include receiving, from the wearabledevice, position information associated with the wearable device. Insome examples, the position information can include a pose of thewearable device. In some cases, the position information can be based onsensor data from one or more sensors on the wearable device. In somecases, the position information can include a measurement from aninertial measurement unit from one or more sensors on the wearabledevice and/or an elevation measured by a pressure sensor from the one ormore sensors.

At block 526, the process 520 can include determining a second positionof the wearable device based on the received position information. Insome cases, the second position can include a position of the wearabledevice relative to the electronic device. In some cases, the secondposition can include a position of the wearable device within acoordinate system, such as a coordinate system of the wearable deviceand/or a coordinate system of the electronic device.

At block 528, the process 520 can include tracking, based on the firstposition and the second position, a movement of the wearable devicerelative to the electronic device.

In some aspects, tracking the movement of the wearable device caninclude determining the first position of the wearable device within afirst coordinate system of the wearable device; transforming the firstcoordinate system of the wearable device to a second coordinate systemof the electronic device; and determining the second position of thewearable device within the second coordinate system of the electronicdevice.

In some aspects, the process 520 can include determining, based on thesecond position of the wearable device and/or the tracked movement ofthe wearable device, whether the wearable device is within a FOV of oneor more image sensors on the electronic device and/or visible to the oneor more image sensors on the electronic device. In some aspects, theprocess 520 can include tracking, based on the second position of thewearable device and/or the tracked movement of the wearable device, alocation of a hand associated with the wearable device. In someexamples, the hand can include a hand wearing the wearable device (e.g.,on a wrist, on a finger, etc.) or holding the wearable device.

In some aspects, the process 520 can include, based on a determinationthat the wearable device is within the FOV of the one or more imagesensors and/or visible to the one or more image sensors, capturing oneor more images of the hand via at least one image sensor from the one ormore image sensors; and tracking the location of the hand based on theone or more images of the hand. In some examples, the location of thehand is tracked relative to a first coordinate system of the wearabledevice.

In some cases, the process 520 can include determining, based on theposition information, that the wearable device is outside of the FOV ofthe one or more image sensors and moving towards an area within the FOVof the one or more image sensors; and based on the determining that thewearable device is outside of the FOV of the one or more image sensorsand moving towards the area within the FOV of the one or more imagesensors, initiating one or more imaging operations and/or one or moretracking operations at the electronic device. In some examples, the oneor more tracking operations can be at least partly based on image datafrom the one or more imaging operations.

In some aspects, the process 520 can include, based on a firstdetermination that the wearable device is within a first FOV of a firstimage sensor on the electronic device and/or a second determination thatthe wearable device is visible to the first image sensor on theelectronic device, adjusting a first setting of the first image sensor.In some cases, the first setting can include a power mode of the firstimage sensor and/or an operating state of the first image sensor. Insome aspects, the process 520 can include, based on a thirddetermination that the wearable device is outside of a second FOV of asecond image sensor on the electronic device and/or a fourthdetermination that the wearable device is not visible to the secondimage sensor on the electronic device, adjusting a second setting of thesecond image sensor. In some examples, the second setting can include apower mode of the second image sensor and/or an operating state of thesecond image sensor.

In some examples, adjusting the first setting of the first image sensorcan include changing the power mode of the first image sensor from afirst power mode to a second power mode including a higher power modethan the first power mode and/or changing the operating state of thefirst image sensor from a first operating state to a second operatingstate including a higher operating state than the first operating state.In some examples, the second operating state can include a higherframerate and/or a higher resolution.

In some examples, adjusting the second setting of the second imagesensor can include changing the power mode of the second image sensorfrom a first power mode to a second power mode including a lower powermode than the first power mode and/or changing the operating state ofthe second image sensor from a first operating state to a secondoperating state including a lower operating state than the firstoperating state. In some cases, the second operating state can include alower framerate and/or a lower resolution.

In some aspects, the process 520 can include, in response to adetermination that the wearable device is outside of the FOV of the oneor more image sensors on the electronic device and/or a view of the oneor more image sensors to the wearable device is obstructed by one ormore objects, tracking a location of the wearable device based onadditional position information from the wearable device. In someaspects, the process 520 can include in response to a determination thatthe wearable device is within the FOV of the one or more image sensorsbut a view of the one or more image sensors to the wearable device isobstructed, tracking a location of the wearable device based onadditional position information from the wearable device.

In some aspects, the process 520 can include, in response to thedetermination that the wearable device is within the FOV of the one ormore image sensors but the view of the one or more image sensors to thewearable device is obstructed, initializing the one or more imagesensors.

In some aspects, the process 520 can include receiving, from thewearable device, an input configured to trigger a privacy mode at theelectronic device; and based on the input configured to trigger theprivacy mode, adjusting an operating state of one or more image sensorsat the electronic device to an off state and/or a disabled state. Insome examples, the input can be based on sensor data from one or moresensors on the wearable device. In some cases, the sensor data canindicate a touch signal corresponding to a touch input at the wearabledevice, a location of the wearable device, and/or a distance between thewearable device and a body part of a user of the wearable device.

In some aspects, the process 520 can include receiving, from thewearable device, an additional input configured to trigger theelectronic device to stop the privacy mode. In some cases, theadditional input can be based on sensor data indicating a touch signalcorresponding to a touch input at the wearable device, a location of thewearable device corresponding to a location of a body part of a user ofthe wearable device, and/or a proximity between the wearable device andthe body part.

In some aspects, the process 520 can include determining, based on datafrom the wearable device and/or a command from the wearable device, oneor more extended reality (XR) inputs to an XR application on theelectronic device. In some examples, the one or more XR inputs caninclude a modification of a virtual element along multiple dimensions inspace, a selection of the virtual element, a navigation event, and/or arequest to measure a distance defined by the first position of thewearable device, the second position of the wearable device, and/or themovement of the wearable device.

In some examples, the virtual element can include a virtual objectrendered by the electronic device, a virtual plane in an environmentrendered by the electronic device, and/or the environment rendered bythe electronic device. In some examples, the navigation event caninclude scrolling rendered content and/or moving from a first interfaceelement to a second interface element.

In some aspects, the process 520 can include receiving, from thewearable device, an input configured to trigger an adjustment of one ormore XR operations at the electronic device. In some examples, the oneor more XR operations can include object detection, objectclassification, object tracking, pose estimation, and/or shapeestimation.

In some examples, the process 500 or the process 520 may be performed byone or more computing devices or apparatuses. In one illustrativeexample, the process 500 can be performed by the XR system 100 and/orthe wearable device 150 shown in FIG. 1 and/or one or more computingdevices with the computing device architecture 600 shown in FIG. 6 . Inanother illustrative example, the process 520 can be performed by the XRsystem 100 and/or one or more computing devices with the computingdevice architecture 600 shown in FIG. 6 . In some cases, such acomputing device or apparatus may include a processor, microprocessor,microcomputer, or other component of a device that is configured tocarry out the steps of the process 500 or the process 520. In someexamples, such computing device or apparatus may include one or moresensors configured to capture image data and/or other sensormeasurements. For example, the computing device can include asmartphone, a head-mounted display, a mobile device, or other suitabledevice. In some examples, such computing device or apparatus may includea camera configured to capture one or more images or videos. In somecases, such computing device may include a display for displayingimages. In some examples, the one or more sensors and/or camera areseparate from the computing device, in which case the computing devicereceives the sensed data. Such computing device may further include anetwork interface configured to communicate data.

The components of the computing device can be implemented in circuitry.For example, the components can include and/or can be implemented usingelectronic circuits or other electronic hardware, which can include oneor more programmable electronic circuits (e.g., microprocessors,graphics processing units (GPUs), digital signal processors (DSPs),central processing units (CPUs), and/or other suitable electroniccircuits), and/or can include and/or be implemented using computersoftware, firmware, or any combination thereof, to perform the variousoperations described herein. The computing device may further include adisplay (as an example of the output device or in addition to the outputdevice), a network interface configured to communicate and/or receivethe data, any combination thereof, and/or other component(s). Thenetwork interface may be configured to communicate and/or receiveInternet Protocol (IP) based data or other type of data.

The process 500 and the process 520 are illustrated as logical flowdiagrams, the operations of which represent a sequence of operationsthat can be implemented in hardware, computer instructions, or acombination thereof. In the context of computer instructions, theoperations represent computer-executable instructions stored on one ormore computer-readable storage media that, when executed by one or moreprocessors, perform the recited operations. Generally,computer-executable instructions include routines, programs, objects,components, data structures, and the like that perform particularfunctions or implement particular data types. The order in which theoperations are described is not intended to be construed as alimitation, and any number of the described operations can be combinedin any order and/or in parallel to implement the processes.

Additionally, the process 500 or the process 520 may be performed underthe control of one or more computer systems configured with executableinstructions and may be implemented as code (e.g., executableinstructions, one or more computer programs, or one or moreapplications) executing collectively on one or more processors, byhardware, or combinations thereof. As noted above, the code may bestored on a computer-readable or machine-readable storage medium, forexample, in the form of a computer program comprising a plurality ofinstructions executable by one or more processors. The computer-readableor machine-readable storage medium may be non-transitory.

FIG. 6 illustrates an example computing device architecture 600 of anexample computing device which can implement various techniquesdescribed herein. For example, the computing device architecture 600 canimplement at least some portions of the XR system 100 shown in FIG. 1 .The components of the computing device architecture 600 are shown inelectrical communication with each other using a connection 605, such asa bus. The example computing device architecture 600 includes aprocessing unit (CPU or processor) 610 and a computing device connection605 that couples various computing device components including thecomputing device memory 615, such as read only memory (ROM) 620 andrandom access memory (RAM) 625, to the processor 610.

The computing device architecture 600 can include a cache of high-speedmemory connected directly with, in close proximity to, or integrated aspart of the processor 610. The computing device architecture 600 cancopy data from the memory 615 and/or the storage device 630 to the cache612 for quick access by the processor 610. In this way, the cache canprovide a performance boost that avoids processor 610 delays whilewaiting for data. These and other modules can control or be configuredto control the processor 610 to perform various actions. Other computingdevice memory 615 may be available for use as well. The memory 615 caninclude multiple different types of memory with different performancecharacteristics. The processor 610 can include any general purposeprocessor and a hardware or software service stored in storage device630 and configured to control the processor 610 as well as aspecial-purpose processor where software instructions are incorporatedinto the processor design. The processor 610 may be a self-containedsystem, containing multiple cores or processors, a bus, memorycontroller, cache, etc. A multi-core processor may be symmetric orasymmetric.

To enable user interaction with the computing device architecture 600,an input device 645 can represent any number of input mechanisms, suchas a microphone for speech, a touch-sensitive screen for gesture orgraphical input, keyboard, mouse, motion input, speech and so forth. Anoutput device 665 can also be one or more of a number of outputmechanisms known to those of skill in the art, such as a display,projector, television, speaker device. In some instances, multimodalcomputing devices can enable a user to provide multiple types of inputto communicate with the computing device architecture 600. Thecommunication interface 640 can generally govern and manage the userinput and computing device output. There is no restriction on operatingon any particular hardware arrangement and therefore the basic featureshere may easily be substituted for improved hardware or firmwarearrangements as they are developed.

Storage device 630 is a non-volatile memory and can be a hard disk orother types of computer readable media which can store data that areaccessible by a computer, such as magnetic cassettes, flash memorycards, solid state memory devices, digital versatile disks, cartridges,random access memories (RAMs) 625, read only memory (ROM) 620, andhybrids thereof. The storage device 630 can include software, code,firmware, etc., for controlling the processor 610. Other hardware orsoftware modules are contemplated. The storage device 630 can beconnected to the computing device connection 605. In one aspect, ahardware module that performs a particular function can include thesoftware component stored in a computer-readable medium in connectionwith the necessary hardware components, such as the processor 610,connection 605, output device 665, and so forth, to carry out thefunction.

The term “computer-readable medium” includes, but is not limited to,portable or non-portable storage devices, optical storage devices, andvarious other mediums capable of storing, containing, or carryinginstruction(s) and/or data. A computer-readable medium may include anon-transitory medium in which data can be stored and that does notinclude carrier waves and/or transitory electronic signals propagatingwirelessly or over wired connections. Examples of a non-transitorymedium may include, but are not limited to, a magnetic disk or tape,optical storage media such as compact disk (CD) or digital versatiledisk (DVD), flash memory, memory or memory devices. A computer-readablemedium may have stored thereon code and/or machine-executableinstructions that may represent a procedure, a function, a subprogram, aprogram, a routine, a subroutine, a module, a software package, a class,or any combination of instructions, data structures, or programstatements. A code segment may be coupled to another code segment or ahardware circuit by passing and/or receiving information, data,arguments, parameters, or memory contents. Information, arguments,parameters, data, etc. may be passed, forwarded, or transmitted via anysuitable means including memory sharing, message passing, token passing,network transmission, or the like.

In some embodiments the computer-readable storage devices, mediums, andmemories can include a cable or wireless signal containing a bit streamand the like. However, when mentioned, non-transitory computer-readablestorage media expressly exclude media such as energy, carrier signals,electromagnetic waves, and signals per se.

Specific details are provided in the description above to provide athorough understanding of the embodiments and examples provided herein.However, it will be understood by one of ordinary skill in the art thatthe embodiments may be practiced without these specific details. Forclarity of explanation, in some instances the present technology may bepresented as including individual functional blocks comprising devices,device components, steps or routines in a method embodied in software,or combinations of hardware and software. Additional components may beused other than those shown in the figures and/or described herein. Forexample, circuits, systems, networks, processes, and other componentsmay be shown as components in block diagram form in order not to obscurethe embodiments in unnecessary detail. In other instances, well-knowncircuits, processes, algorithms, structures, and techniques may be shownwithout unnecessary detail in order to avoid obscuring the embodiments.

Individual embodiments may be described above as a process or methodwhich is depicted as a flowchart, a flow diagram, a data flow diagram, astructure diagram, or a block diagram. Although a flowchart may describethe operations as a sequential process, many of the operations can beperformed in parallel or concurrently. In addition, the order of theoperations may be re-arranged. A process is terminated when itsoperations are completed, but could have additional steps not includedin a figure. A process may correspond to a method, a function, aprocedure, a subroutine, a subprogram, etc. When a process correspondsto a function, its termination can correspond to a return of thefunction to the calling function or the main function.

Processes and methods according to the above-described examples can beimplemented using computer-executable instructions that are stored orotherwise available from computer-readable media. Such instructions caninclude, for example, instructions and data which cause or otherwiseconfigure a general purpose computer, special purpose computer, or aprocessing device to perform a certain function or group of functions.Portions of computer resources used can be accessible over a network.The computer executable instructions may be, for example, binaries,intermediate format instructions such as assembly language, firmware,source code. Examples of computer-readable media that may be used tostore instructions, information used, and/or information created duringmethods according to described examples include magnetic or opticaldisks, flash memory, USB devices provided with non-volatile memory,networked storage devices, and so on.

Devices implementing processes and methods according to thesedisclosures can include hardware, software, firmware, middleware,microcode, hardware description languages, or any combination thereof,and can take any of a variety of form factors. When implemented insoftware, firmware, middleware, or microcode, the program code or codesegments to perform the necessary tasks (e.g., a computer-programproduct) may be stored in a computer-readable or machine-readablemedium. A processor(s) may perform the necessary tasks. Typical examplesof form factors include laptops, smart phones, mobile phones, tabletdevices or other small form factor personal computers, personal digitalassistants, rackmount devices, standalone devices, and so on.Functionality described herein also can be embodied in peripherals oradd-in cards. Such functionality can also be implemented on a circuitboard among different chips or different processes executing in a singledevice, by way of further example.

The instructions, media for conveying such instructions, computingresources for executing them, and other structures for supporting suchcomputing resources are example means for providing the functionsdescribed in the disclosure.

In the foregoing description, aspects of the application are describedwith reference to specific embodiments thereof, but those skilled in theart will recognize that the application is not limited thereto. Thus,while illustrative embodiments of the application have been described indetail herein, it is to be understood that the inventive concepts may beotherwise variously embodied and employed, and that the appended claimsare intended to be construed to include such variations, except aslimited by the prior art. Various features and aspects of theabove-described application may be used individually or jointly.Further, embodiments can be utilized in any number of environments andapplications beyond those described herein without departing from thebroader spirit and scope of the specification. The specification anddrawings are, accordingly, to be regarded as illustrative rather thanrestrictive. For the purposes of illustration, methods were described ina particular order. It should be appreciated that in alternateembodiments, the methods may be performed in a different order than thatdescribed.

One of ordinary skill will appreciate that the less than (“<”) andgreater than (“>”) symbols or terminology used herein can be replacedwith less than or equal to (“≤”) and greater than or equal to (“≥”)symbols, respectively, without departing from the scope of thisdescription.

Where components are described as being “configured to” perform certainoperations, such configuration can be accomplished, for example, bydesigning electronic circuits or other hardware to perform theoperation, by programming programmable electronic circuits (e.g.,microprocessors, or other suitable electronic circuits) to perform theoperation, or any combination thereof.

The phrase “coupled to” refers to any component that is physicallyconnected to another component either directly or indirectly, and/or anycomponent that is in communication with another component (e.g.,connected to the other component over a wired or wireless connection,and/or other suitable communication interface) either directly orindirectly.

Claim language or other language in the disclosure reciting “at leastone of” a set and/or “one or more” of a set indicates that one member ofthe set or multiple members of the set (in any combination) satisfy theclaim. For example, claim language reciting “at least one of A and B” or“at least one of A or B” means A, B, or A and B. In another example,claim language reciting “at least one of A, B, and C” or “at least oneof A, B, or C” means A, B, C, or A and B, or A and C, or B and C, or Aand B and C. The language “at least one of” a set and/or “one or more”of a set does not limit the set to the items listed in the set. Forexample, claim language reciting “at least one of A and B” or “at leastone of A or B” can mean A, B, or A and B, and can additionally includeitems not listed in the set of A and B.

The various illustrative logical blocks, modules, circuits, andalgorithm steps described in connection with the examples disclosedherein may be implemented as electronic hardware, computer software,firmware, or combinations thereof. To clearly illustrate thisinterchangeability of hardware and software, various illustrativecomponents, blocks, modules, circuits, and steps have been describedabove generally in terms of their functionality. Whether suchfunctionality is implemented as hardware or software depends upon theparticular application and design constraints imposed on the overallsystem. Skilled artisans may implement the described functionality invarying ways for each particular application, but such implementationdecisions should not be interpreted as causing a departure from thescope of the present application.

The techniques described herein may also be implemented in electronichardware, computer software, firmware, or any combination thereof. Suchtechniques may be implemented in any of a variety of devices such asgeneral purposes computers, wireless communication device handsets, orintegrated circuit devices having multiple uses including application inwireless communication device handsets and other devices. Any featuresdescribed as modules or components may be implemented together in anintegrated logic device or separately as discrete but interoperablelogic devices. If implemented in software, the techniques may berealized at least in part by a computer-readable data storage mediumcomprising program code including instructions that, when executed,performs one or more of the methods, algorithms, and/or operationsdescribed above. The computer-readable data storage medium may form partof a computer program product, which may include packaging materials.The computer-readable medium may comprise memory or data storage media,such as random access memory (RAM) such as synchronous dynamic randomaccess memory (SDRAM), read-only memory (ROM), non-volatile randomaccess memory (NVRAM), electrically erasable programmable read-onlymemory (EEPROM), FLASH memory, magnetic or optical data storage media,and the like. The techniques additionally, or alternatively, may berealized at least in part by a computer-readable communication mediumthat carries or communicates program code in the form of instructions ordata structures and that can be accessed, read, and/or executed by acomputer, such as propagated signals or waves.

The program code may be executed by a processor, which may include oneor more processors, such as one or more digital signal processors(DSPs), general purpose microprocessors, an application specificintegrated circuits (ASICs), field programmable logic arrays (FPGAs), orother equivalent integrated or discrete logic circuitry. Such aprocessor may be configured to perform any of the techniques describedin this disclosure. A general purpose processor may be a microprocessor;but in the alternative, the processor may be any conventional processor,controller, microcontroller, or state machine. A processor may also beimplemented as a combination of computing devices, e.g., a combinationof a DSP and a microprocessor, a plurality of microprocessors, one ormore microprocessors in conjunction with a DSP core, or any other suchconfiguration. Accordingly, the term “processor,” as used herein mayrefer to any of the foregoing structure, any combination of theforegoing structure, or any other structure or apparatus suitable forimplementation of the techniques described herein.

Illustrative examples of the disclosure include:

Aspect 1. A wearable device comprising: a structure defining a receivingspace configured to receive a body part associated with a user, thestructure comprising an engagement surface configured to contact thebody part received via the receiving space; one or more sensorsintegrated with the structure, the one or more sensors being configuredto obtain one or more tracking measurements associated with the wearabledevice, the one or more tracking measurements comprising at least one ofa position of the structure in a physical space and movement of thestructure; and a wireless transmitter configured to send, to an extendedreality device, at least one of the position of the structure in thephysical space and the movement of the structure.

Aspect 2. The wearable device of Aspect 1, wherein the one or moretracking measurements further comprise at least one of a distance of thestructure relative to one or more objects, a velocity vector indicatinga velocity of the movement of the structure, and an elevation of thestructure in the physical space.

Aspect 3. The wearable device of any of Aspects 1 to 2, wherein the oneor more objects comprise at least one of the XR device, a different bodypart associated with the user, and an input device.

Aspect 4. The wearable device of any of Aspects 1 to 3, wherein the bodypart associated with the user comprises at least one of a finger, ahand, and a wrist, and wherein the different body part associated withthe user comprises at least one of a different finger, a different hand,and a different wrist.

Aspect 5. The wearable device of any of Aspects 1 to 4, wherein thewearable device is configured to send, via the wireless transmitter andto the XR device, an input configured to trigger a privacy mode at theXR device, and wherein the privacy mode comprises an operating statewith one or more image sensors at the XR device being at least one ofturned off and disabled.

Aspect 6. The wearable device of any of Aspects 1 to 5, wherein theinput is based on one or more measurements from the one or more sensors,and wherein the one or more measurements indicate at least one of atouch signal corresponding to a touch input at the wearable device, afirst location of the wearable device that differs from a secondlocation of the body part, and a distance between the wearable deviceand the body part.

Aspect 7. The wearable device of any of Aspects 1 to 6, wherein thewearable device is configured to send, via the wireless transmitter andto the XR device, an additional input configured to trigger the XRdevice to stop the privacy mode, wherein the input is based on sensordata indicating at least one of a touch signal corresponding to a touchinput at the wearable device, a location of the wearable devicecorresponding to a location of the body part, and a proximity betweenthe wearable device and the body part.

Aspect 8. The wearable device of any of Aspects 1 to 7, wherein thewearable device is configured to send, via the wireless transmitter andto the XR device, an XR input associated with an XR application at theXR device, the XR input being based on one or more measurements from theone or more sensors.

Aspect 9. The wearable device of any of Aspects 1 to 8, wherein thewearable device is configured to send, via the wireless transmitter andto the XR device, an input configured to trigger an adjustment of atleast one of a device setting at the XR device and one or more XRoperations at the XR device.

Aspect 10. The wearable device of any of Aspects 1 to 9, wherein thedevice setting comprises at least one of a power mode associated withone or more image sensors, a framerate associated with one or more imagesensors, and a resolution associated with one or more image sensors, andwherein the one or more XR operations comprise at least one of objectdetection, object classification, pose estimation, and shape estimation.

Aspect 11. The wearable device of any of Aspects 1 to 10, wherein theone or more sensors comprise at least one of an accelerometer, agyroscope, a pressure sensor, an audio sensor, a touch sensor, and amagnetometer.

Aspect 12. The wearable device of any of Aspects 1 to 11, wherein thewearable device comprises a ring, and wherein the body part comprises afinger of the user.

Aspect 13. The wearable device of any of Aspects 1 to 11, wherein thewearable device comprises a bracelet, and wherein the body partcomprises a wrist of the user.

Aspect 14. The wearable device of any of Aspects 1 to 11, wherein thewearable device comprises a glove, and wherein the body part comprises ahand of the user.

Aspect 15. A method for processing tracking data, the method comprising:establishing a wireless connection between a wearable device and anextended reality device, the wearable device comprising a structuredefining a receiving space configured to receive a body part associatedwith a user, the structure comprising a surface configured to contactthe body part received via the receiving space; obtaining, via one ormore sensors integrated with the structure associated with the wearabledevice, one or more tracking measurements associated with the wearabledevice, the one or more tracking measurements comprising at least one ofa position of the structure in a physical space and movement of thestructure; and sending, via a wireless transmitter of the wearabledevice to the XR device, at least one of the position of the structurein the physical space and the movement of the structure.

Aspect 16. The method of Aspect 15, wherein the one or more trackingmeasurements further comprise at least one of a distance of thestructure relative to one or more objects, a velocity vector indicatinga velocity of the movement of the structure, and an elevation of thestructure in the physical space.

Aspect 17. The method of any of Aspects 15 to 16, wherein the one ormore objects comprise at least one of the XR device, a second body partassociated with the user, and an input device.

Aspect 18. The method of any of Aspects 15 to 17, wherein the body partassociated with the user comprises at least one of a finger, a hand, anda wrist, and wherein the second body part associated with the usercomprises at least one of a second finger, a second hand, and a secondwrist.

Aspect 19. The method of any of Aspects 15 to 18, further comprisingsending, via the wireless transmitter of the wearable device to the XRdevice, an input configured to trigger a privacy mode at the XR device,wherein the privacy mode comprises an operating state with one or moreimage sensors at the XR device being at least one of turned off anddisabled.

Aspect 20. The method of any of Aspects 15 to 19, wherein the input isbased on one or more measurements from the one or more sensors, andwherein the one or more measurements indicate at least one of a touchsignal corresponding to a touch input at the wearable device, a firstlocation of the wearable that differs from a second location of the bodypart, and a distance between the wearable and the body part.

Aspect 21. The method of any of Aspects 15 to 20, further comprisingsending, via the wireless transmitter of the wearable device to the XRdevice, an additional input configured to trigger the XR device to stopthe privacy mode, wherein the input is based on sensor data indicatingat least one of a touch signal corresponding to a touch input at thewearable device, a location of the wearable device corresponding to alocation of the body part, and a proximity between the wearable deviceand the body part.

Aspect 22. The method of any of Aspects 15 to 21, further comprisingsending, via the wireless transmitter of the wearable device to the XRdevice, an XR input associated with an XR application at the XR device,the XR input being based on one or more measurements from the one ormore sensors.

Aspect 23. The method of any of Aspects 15 to 22, further comprisingsending, via the wireless transmitter of the wearable device to the XRdevice, an input configured to trigger an adjustment of at least one ofa device setting at the XR device and one or more XR operations at theXR device.

Aspect 24. The method of any of Aspects 15 to 23, wherein the devicesetting comprises at least one of a power mode associated with one ormore image sensors, a framerate associated with one or more imagesensors, and a resolution associated with one or more image sensors, andwherein the one or more XR operations comprise at least one of objectdetection, object classification, pose estimation, and shape estimation.

Aspect 25. The method of any of Aspects 15 to 24, wherein the wearabledevice comprises a ring, and wherein the body part comprises a finger ofthe user.

Aspect 26. The method of any of Aspects 15 to 25, wherein the wearabledevice comprises a bracelet, and wherein the body part comprises a wristof the user.

Aspect 27. The method of any of Aspects 15 to 26, wherein the wearabledevice comprises a glove, and wherein the body part comprises a hand ofthe user.

Aspect 28. The method of any of Aspects 15 to 27, wherein the one ormore sensors comprise at least one of an accelerometer, a gyroscope, apressure sensor, an audio sensor, a touch sensor, and a magnetometer.

Aspect 29. A non-transitory computer-readable medium having storedthereon instructions which, when executed by one or more processors,cause the one or more processors to perform a method according to any ofAspects 15 to 27.

Aspect 30. A wearable device comprising means for performing a methodaccording to any of Aspects 15 to 27.

Aspect 31. The wearable device of Aspect 30, wherein the one or moresensors comprise at least one of an accelerometer, a gyroscope, apressure sensor, an audio sensor, a touch sensor, and a magnetometer.

Aspect 32. The wearable device of Aspect 30 or 31, wherein the wearabledevice comprises a glove, and wherein the body part comprises a hand ofthe user.

Aspect 33. The wearable device of Aspect 30 or 31, wherein the wearabledevice comprises a ring, and wherein the body part comprises a finger ofthe user.

Aspect 34. The wearable device of Aspect 30 or 31, wherein the wearabledevice comprises a bracelet, and wherein the body part comprises a wristof the user.

Aspect 35. A method for processing tracking data, the method comprising:establishing a wireless connection between a wearable device and anextended reality (XR) device, the wearable device comprising a structuredefining a receiving space configured to receive a body part associatedwith a user, the structure comprising a surface configured to contactthe body part received via the receiving space; receiving, from thewearable device, one or more tracking measurements calculated by one ormore sensors of the wearable device, the one or more trackingmeasurements comprising at least one of a position of the structure in aphysical space and movement of the structure; and translating, by the XRdevice, the one or more tracking measurements into one or more XR inputsto an XR application at the XR device.

Aspect 36. The method of Aspect 35, wherein the one or more trackingmeasurements further comprise at least one of a distance of thestructure relative to one or more objects, a velocity vector indicatinga velocity of the movement of the structure, and an elevation of thestructure in the physical space.

Aspect 37. The method of Aspect 36, wherein the one or more objectscomprise at least one of the XR device, a second body part associatedwith the user, and an input device.

Aspect 38. The method of Aspect 37, wherein the body part associatedwith the user comprises at least one of a finger, a hand, and a wrist,and wherein the second body part associated with the user comprises atleast one of a second finger, a second hand, and a second wrist.

Aspect 39. The method of any of Aspects 35 to 37, wherein the one ormore XR inputs comprise at least one of a request to measure a distanceddefined by the at least one of the position of the structure in thephysical space and the movement of the structure, a modification of avirtual element along multiple dimensions in space, a selection of thevirtual element, and a navigation event.

Aspect 40. The method of Aspect 39, wherein the virtual elementcomprises at least one of a virtual object rendered by the XR device, avirtual plane in an environment rendered by the XR device, and theenvironment rendered by the XR device.

Aspect 41. The method of Aspect 39, wherein the navigation eventcomprises at least one of scrolling rendered content and moving from afirst interface element to a second interface element.

Aspect 42. The method of any of Aspects 35 to 41, further comprisingreceiving, from the wearable device, an input configured to trigger aprivacy mode at the XR device, wherein the privacy mode comprises anoperating state with one or more image sensors at the XR device being atleast one of turned off and disabled.

Aspect 43. The method of Aspect 42, wherein the input is based on one ormore measurements from the one or more sensors, and wherein the one ormore measurements indicate at least one of a touch signal correspondingto a touch input at the wearable device, a first location of thewearable that differs from a second location of the body part, and adistance between the wearable and the body part.

Aspect 44. The method of Aspect 42, further comprising receiving, fromthe wearable device to the XR device, an additional input configured totrigger the XR device to stop the privacy mode, wherein the input isbased on sensor data indicating at least one of a touch signalcorresponding to a touch input at the wearable device, a location of thewearable device corresponding to a location of the body part, and aproximity between the wearable device and the body part.

Aspect 45. The method of any of Aspects 35 to 44, further comprisingreceiving, from the wearable device, an input configured to trigger anadjusted power mode at the XR device, wherein the adjusted power modecomprises a lower power state relative to a power state prior to theadjusted power mode.

Aspect 46. The method of any of Aspects 35 to 45, further comprisingreceiving, from the wearable device to the XR device, an inputconfigured to trigger an adjustment of at least one of a device settingat the XR device and one or more XR operations at the XR device.

Aspect 47. The method of Aspect 46, wherein the device setting comprisesat least one of a power mode associated with one or more image sensors,a framerate associated with one or more image sensors, and a resolutionassociated with one or more image sensors.

Aspect 48. The method of Aspect 46 or 47, wherein the one or more XRoperations comprise at least one of object detection, objectclassification, pose estimation, and shape estimation.

Aspect 49. The method of any of Aspects 35 to 48, wherein the one ormore sensors comprise at least one of an accelerometer, a gyroscope, apressure sensor, an audio sensor, a touch sensor, and a magnetometer.

Aspect 50. The method of any of Aspects 35 to 49, wherein the wearabledevice comprises a bracelet, and wherein the body part comprises a wristof the user.

Aspect 51. The method of any of Aspects 35 to 49, wherein the wearabledevice comprises a ring, and wherein the body part comprises a finger ofthe user.

Aspect 52. The method of any of Aspects 35 to 49, wherein the wearabledevice comprises a glove, and wherein the body part comprises a hand ofthe user.

Aspect 53. An apparatus for processing tracking data, the apparatuscomprising: memory; and one or more processors coupled to the memory,the one or more processors being configured to: establish a wirelessconnection between a wearable device and an extended reality (XR)device, the wearable device comprising a structure defining a receivingspace configured to receive a body part associated with a user, thestructure comprising a surface configured to contact the body partreceived via the receiving space; receive, from the wearable device, oneor more tracking measurements calculated by one or more sensors of thewearable device, the one or more tracking measurements comprising atleast one of a position of the structure in a physical space andmovement of the structure; and translate, by the XR device, the one ormore tracking measurements into one or more XR inputs to an XRapplication at the XR device.

Aspect 54. The apparatus of Aspect 53, wherein the one or more trackingmeasurements further comprise at least one of a distance of thestructure relative to one or more objects, a velocity vector indicatinga velocity of the movement of the structure, and an elevation of thestructure in the physical space.

Aspect 55. The apparatus of Aspect 54, wherein the one or more objectscomprise at least one of the XR device, a second body part associatedwith the user, and an input device.

Aspect 56. The apparatus of Aspect 55, wherein the body part associatedwith the user comprises at least one of a finger, a hand, and a wrist,and wherein the second body part associated with the user comprises atleast one of a second finger, a second hand, and a second wrist.

Aspect 57. The apparatus of any of Aspects 53 to 55, wherein the one ormore XR inputs comprise at least one of a request to measure a distanceddefined by the at least one of the position of the structure in thephysical space and the movement of the structure, a modification of avirtual element along multiple dimensions in space, a selection of thevirtual element, and a navigation event.

Aspect 58. The apparatus of Aspect 57, wherein the virtual elementcomprises at least one of a virtual object rendered by the XR device, avirtual plane in an environment rendered by the XR device, and theenvironment rendered by the XR device.

Aspect 59. The apparatus of Aspect 57, wherein the navigation eventcomprises at least one of scrolling rendered content and moving from afirst interface element to a second interface element.

Aspect 60. The apparatus of any of Aspects 53 to 59, further comprisingreceiving, from the wearable device, an input configured to trigger aprivacy mode at the XR device, wherein the privacy mode comprises anoperating state with one or more image sensors at the XR device being atleast one of turned off and disabled.

Aspect 61. The apparatus of Aspect 60, wherein the input is based on oneor more measurements from the one or more sensors, and wherein the oneor more measurements indicate at least one of a touch signalcorresponding to a touch input at the wearable device, a first locationof the wearable that differs from a second location of the body part,and a distance between the wearable and the body part.

Aspect 62. The apparatus of Aspect 60, further comprising receiving,from the wearable device to the XR device, an additional inputconfigured to trigger the XR device to stop the privacy mode, whereinthe input is based on sensor data indicating at least one of a touchsignal corresponding to a touch input at the wearable device, a locationof the wearable device corresponding to a location of the body part, anda proximity between the wearable device and the body part.

Aspect 63. The apparatus of any of Aspects 53 to 62, further comprisingreceiving, from the wearable device, an input configured to trigger anadjusted power mode at the XR device, wherein the adjusted power modecomprises a lower power state relative to a power state prior to theadjusted power mode.

Aspect 64. The apparatus of any of Aspects 53 to 63, further comprisingreceiving, from the wearable device to the XR device, an inputconfigured to trigger an adjustment of at least one of a device settingat the XR device and one or more XR operations at the XR device.

Aspect 65. The apparatus of Aspect 64, wherein the device settingcomprises at least one of a power mode associated with one or more imagesensors, a framerate associated with one or more image sensors, and aresolution associated with one or more image sensors.

Aspect 66. The apparatus of Aspect 64 or 65, wherein the one or more XRoperations comprise at least one of object detection, objectclassification, pose estimation, and shape estimation.

Aspect 67. The apparatus of any of Aspects 53 to 66, wherein the one ormore sensors comprise at least one of an accelerometer, a gyroscope, apressure sensor, an audio sensor, a touch sensor, and a magnetometer.

Aspect 68. The apparatus of any of Aspects 53 to 67, wherein thewearable device comprises a bracelet, and wherein the body partcomprises a wrist of the user.

Aspect 69. The apparatus of any of Aspects 53 to 67, wherein thewearable device comprises a ring, and wherein the body part comprises afinger of the user.

Aspect 70. The apparatus of any of Aspects 53 to 67, wherein thewearable device comprises a glove, and wherein the body part comprises ahand of the user.

Aspect 71. The apparatus of any of Aspects 53 to 67, wherein theapparatus comprises a mobile device.

Aspect 72. The apparatus of any of Aspects 53 to 67, wherein theapparatus comprises a camera.

Aspect 73. The apparatus of any of Aspects 53 to 67, wherein theapparatus comprises the XR device and a display.

Aspect 74. A non-transitory computer-readable medium having storedthereon instructions which, when executed by one or more processors,cause the one or more processors to perform a method according to any ofAspects 35 to 52.

Aspect 75. An apparatus comprising means for performing a methodaccording to any of Aspects 35 to 52.

Aspect 76. An apparatus comprising: memory; and one or more processorscoupled to the memory, the one or more processors being configured to:determine a first position of a wearable device in a physical space;receive, from the wearable device, position information associated withthe wearable device; determine a second position of the wearable devicebased on the received position information; and track, based on thefirst position and the second position, a movement of the wearabledevice relative to the apparatus.

Aspect 77. The apparatus of Aspect 76, wherein, to track the movement ofthe wearable device, the one or more processors are configured to:determine the first position of the wearable device within a firstcoordinate system of the wearable device; transform the first coordinatesystem of the wearable device to a second coordinate system of theapparatus; and determine the second position of the wearable devicewithin the second coordinate system of the apparatus.

Aspect 78. The apparatus of any of Aspects 76 to 77, wherein the one ormore processors are configured to: determine, based on at least one ofthe second position of the wearable device and the tracked movement ofthe wearable device, whether the wearable device is at least one ofwithin a field-of-view (FOV) of one or more image sensors on theapparatus and visible to the one or more image sensors on the apparatus.

Aspect 79. The apparatus of Aspect 78, wherein the one or moreprocessors are configured to: track, based on at least one of the secondposition of the wearable device and the tracked movement of the wearabledevice, a location of a hand associated with the wearable device.

Aspect 80. The apparatus of Aspect 79, wherein the one or moreprocessors are configured to: based on a determination that the wearabledevice is within the FOV of the one or more image sensors and visible tothe one or more image sensors, capture one or more images of the handvia at least one image sensor from the one or more image sensors; andtrack the location of the hand further based on the one or more imagesof the hand, the location of the hand being tracked relative to a firstcoordinate system of the wearable device.

Aspect 81. The apparatus of Aspect 78, wherein the one or moreprocessors are configured to: determine, based on the positioninformation, that the wearable device is outside of the FOV of the oneor more image sensors and moving towards an area within the FOV of theone or more image sensors; and based on the determining that thewearable device is outside of the FOV of the one or more image sensorsand moving towards the area within the FOV of the one or more imagesensors, initiate one or more imaging operations and one or moretracking operations at the apparatus, the one or more trackingoperations being at least partly based on image data from the one ormore imaging operations.

Aspect 82. The apparatus of Aspect 78, wherein the one or moreprocessors are configured to: based on at least one of a firstdetermination that the wearable device is within a first FOV of a firstimage sensor on the apparatus and a second determination that thewearable device is visible to the first image sensor on the apparatus,adjust a first setting of the first image sensor, the first settingcomprising at least one of a power mode of the first image sensor and anoperating state of the first image sensor; and based on at least one ofa third determination that the wearable device is outside of a secondFOV of a second image sensor on the apparatus and a fourth determinationthat the wearable device is not visible to the second image sensor onthe apparatus, adjust a second setting of the second image sensor, thesecond setting comprising at least one of a power mode of the secondimage sensor and an operating state of the second image sensor.

Aspect 83. The apparatus of Aspect 82, wherein, to adjust the firstsetting of the first image sensor, the one or more processors areconfigured to change at least one of the power mode of the first imagesensor from a first power mode to a second power mode comprising ahigher power mode than the first power mode and the operating state ofthe first image sensor from a first operating state to a secondoperating state comprising a higher operating state than the firstoperating state, the second operating state comprising at least one of ahigher framerate and a higher resolution.

Aspect 84. The apparatus of Aspect 82, wherein, to adjust the secondsetting of the second image sensor, the one or more processors areconfigured to change at least one of the power mode of the second imagesensor from a first power mode to a second power mode comprising a lowerpower mode than the first power mode and the operating state of thesecond image sensor from a first operating state to a second operatingstate comprising a lower operating state than the first operating state,the second operating state comprising at least one of a lower framerateand a lower resolution.

Aspect 85. The apparatus of Aspect 78, wherein the one or moreprocessors are configured to: in response to a determination that thewearable device is not visible to the one or more image sensors on theapparatus, track a location of the wearable device based on additionalposition information from the wearable device.

Aspect 86. The apparatus of Aspect 78, wherein the one or moreprocessors are configured to: in response to a determination that thewearable device is within the FOV of the one or more image sensors and aview of the one or more image sensors to the wearable device isobstructed, track a location of the wearable device based on additionalposition information from the wearable device.

Aspect 87. The apparatus of Aspect 86, wherein the one or moreprocessors are configured to: in response to the determination that thewearable device is within the FOV of the one or more image sensors andthe view of the one or more image sensors to the wearable device isobstructed, initialize the one or more image sensors.

Aspect 88. The apparatus of any of Aspects 76 to 87, wherein, todetermine the first position of the wearable device, the one or moreprocessors are configured to: receive, from the wearable device, atleast one of image data from one or more image sensors on the apparatusand data associated with one or more measurements from one or moresensors on the wearable device; and determine the first position of thewearable device based on at least one of the image data from the one ormore image sensors and data associated with the one or more measurementsfrom the one or more sensors.

Aspect 89. The apparatus of Aspect 88, wherein the data comprises atleast one of a distance of the wearable device relative to one or moreobjects, a velocity vector indicating a velocity of the movement of thewearable device, a touch signal measured by a touch sensor from the oneor more sensors, audio data from an audio sensor from the one or moresensors, and an elevation of the wearable device in the physical space,and wherein the one or more objects comprise at least one of theapparatus, a body part associated with a user of the wearable device,and an input device.

Aspect 90. The apparatus of any of Aspects 76 to 89, wherein the one ormore processors are configured to: receive, from the wearable device, aninput configured to trigger a privacy mode at the apparatus; and basedon the input configured to trigger the privacy mode, adjust an operatingstate of one or more image sensors at the apparatus to at least one ofan off state and a disabled state.

Aspect 91. The apparatus of Aspect 90, wherein the input is based onsensor data from one or more sensors on the wearable device, and whereinthe sensor data indicates at least one of a touch signal correspondingto a touch input at the wearable device, a location of the wearabledevice, and a distance between the wearable device and a body part of auser of the wearable device.

Aspect 92. The apparatus of any of Aspects 90 to 91, wherein the one ormore processors are configured to: receive, from the wearable device, anadditional input configured to trigger the apparatus to stop the privacymode, wherein the additional input is based on sensor data indicating atleast one of a touch signal corresponding to a touch input at thewearable device, a location of the wearable device corresponding to alocation of a body part of a user of the wearable device, and aproximity between the wearable device and the body part.

Aspect 93. The apparatus of any of Aspects 76 to 92, wherein the one ormore processors are configured to: determine, based on at least one ofdata from the wearable device and a command from the wearable device,one or more extended reality (XR) inputs to an XR application on theapparatus.

Aspect 94. The apparatus of Aspect 93, wherein the one or more XR inputscomprise at least one of a modification of a virtual element alongmultiple dimensions in space, a selection of the virtual element, anavigation event, and a request to measure a distance defined by atleast one of the first position of the wearable device, the secondposition of the wearable device, and the movement of the wearabledevice.

Aspect 95. The apparatus of Aspect 94, wherein the virtual elementcomprises at least one of a virtual object rendered by the apparatus, avirtual plane in an environment rendered by the apparatus, and theenvironment rendered by the apparatus.

Aspect 96. The apparatus of any of Aspects 94 to 95, wherein thenavigation event comprises at least one of scrolling rendered contentand moving from a first interface element to a second interface element.

Aspect 97. The apparatus of any of Aspects 76 to 96, wherein the one ormore processors are configured to: receive, from the wearable device, aninput configured to trigger an adjustment of one or more XR operationsat the apparatus, wherein the one or more XR operations comprise atleast one of object detection, object classification, object tracking,pose estimation, and shape estimation.

Aspect 98. The apparatus of any of Aspects 76 to 97, wherein thewearable device comprises a bracelet, a ring, or a glove, and whereinthe position information comprises at least one of a measurement from aninertial measurement unit from one or more sensors on the wearabledevice and an elevation measured by a pressure sensor from the one ormore sensors.

Aspect 99. The apparatus of any of Aspects 76 to 98, wherein theapparatus comprises a mobile device.

Aspect 100. The apparatus of any of Aspects 76 to 99, wherein theapparatus comprises a camera.

Aspect 101. The apparatus of any of Aspects 76 to 100, wherein theapparatus comprises an XR device and a display.

Aspect 102. A method comprising: determining a first position of awearable device in a physical space; receiving, from the wearabledevice, position information associated with the wearable device;determining a second position of the wearable device based on thereceived position information; and tracking, based on the first positionand the second position, a movement of the wearable device relative toan electronic device.

Aspect 103. The method of Aspect 102, wherein tracking the movement ofthe wearable device further comprises: determining the first position ofthe wearable device within a first coordinate system of the wearabledevice; transforming the first coordinate system of the wearable deviceto a second coordinate system of the electronic device; and determiningthe second position of the wearable device within the second coordinatesystem of the electronic device.

Aspect 104. The method of any of Aspects 102 to 103, further comprising:determining, based on at least one of the second position of thewearable device and the tracked movement of the wearable device, whetherthe wearable device is at least one of within a field-of-view (FOV) ofone or more image sensors on the electronic device and visible to theone or more image sensors on the electronic device.

Aspect 105. The method of Aspect 104, further comprising: tracking,based on at least one of the second position of the wearable device andthe tracked movement of the wearable device, a location of a handassociated with the wearable device.

Aspect 106. The method of Aspect 105, further comprising: based on adetermination that the wearable device is within the FOV of the one ormore image sensors and visible to the one or more image sensors,capturing one or more images of the hand via at least one image sensorfrom the one or more image sensors; and tracking the location of thehand further based on the one or more images of the hand, the locationof the hand being tracked relative to a first coordinate system of thewearable device.

Aspect 107. The method of Aspect 104, further comprising: determining,based on the position information, that the wearable device is outsideof the FOV of the one or more image sensors and moving towards an areawithin the FOV of the one or more image sensors; and based on thedetermining that the wearable device is outside of the FOV of the one ormore image sensors and moving towards the area within the FOV of the oneor more image sensors, initiating one or more imaging operations and oneor more tracking operations at the electronic device, the one or moretracking operations being at least partly based on image data from theone or more imaging operations.

Aspect 108. The method of Aspect 104, further comprising: based on atleast one of a first determination that the wearable device is within afirst FOV of a first image sensor on the electronic device and a seconddetermination that the wearable device is visible to the first imagesensor on the electronic device, adjusting a first setting of the firstimage sensor, the first setting comprising at least one of a power modeof the first image sensor and an operating state of the first imagesensor; and based on at least one of a third determination that thewearable device is outside of a second FOV of a second image sensor onthe electronic device and a fourth determination that the wearabledevice is not visible to the second image sensor on the electronicdevice, adjusting a second setting of the second image sensor, thesecond setting comprising at least one of a power mode of the secondimage sensor and an operating state of the second image sensor.

Aspect 109. The method of Aspect 108, wherein adjusting the firstsetting of the first image sensor further comprises changing at leastone of the power mode of the first image sensor from a first power modeto a second power mode comprising a higher power mode than the firstpower mode and the operating state of the first image sensor from afirst operating state to a second operating state comprising a higheroperating state than the first operating state, the second operatingstate comprising at least one of a higher framerate and a higherresolution.

Aspect 110. The method of Aspect 108, wherein adjusting the secondsetting of the second image sensor further comprises changing at leastone of the power mode of the second image sensor from a first power modeto a second power mode comprising a lower power mode than the firstpower mode and the operating state of the second image sensor from afirst operating state to a second operating state comprising a loweroperating state than the first operating state, the second operatingstate comprising at least one of a lower framerate and a lowerresolution.

Aspect 111. The method of Aspect 104, further comprising: in response toa determination that the wearable device is not visible to the one ormore image sensors on the electronic device, tracking a location of thewearable device based on additional position information from thewearable device.

Aspect 112. The method of Aspect 104, further comprising: in response toa determination that the wearable device is within the FOV of the one ormore image sensors and a view of the one or more image sensors to thewearable device is obstructed, tracking a location of the wearabledevice based on additional position information from the wearabledevice.

Aspect 113. The method of Aspect 112, further comprising: in response tothe determination that the wearable device is within the FOV of the oneor more image sensors and the view of the one or more image sensors tothe wearable device is obstructed, initializing the one or more imagesensors.

Aspect 114. The method of any of Aspects 102 to 113, wherein determiningthe first position of the wearable device further comprises: receiving,from the wearable device, at least one of image data from one or moreimage sensors on the electronic device and data associated with one ormore measurements from one or more sensors on the wearable device; anddetermining the first position of the wearable device based on at leastone of the image data from the one or more image sensors and dataassociated with the one or more measurements from the one or moresensors.

Aspect 115. The method of Aspect 114, wherein the data comprises atleast one of a distance of the wearable device relative to one or moreobjects, a velocity vector indicating a velocity of the movement of thewearable device, a touch signal measured by a touch sensor from the oneor more sensors, audio data from an audio sensor from the one or moresensors, and an elevation of the wearable device in the physical space,and wherein the one or more objects comprise at least one of theelectronic device, a body part associated with a user of the wearabledevice, and an input device.

Aspect 116. The method of any of Aspects 102 to 115, further comprising:receiving, from the wearable device, an input configured to trigger aprivacy mode at the electronic device; and based on the input configuredto trigger the privacy mode, adjusting an operating state of one or moreimage sensors at the electronic device to at least one of an off stateand a disabled state.

Aspect 117. The method of Aspect 116, wherein the input is based onsensor data from one or more sensors on the wearable device, and whereinthe sensor data indicates at least one of a touch signal correspondingto a touch input at the wearable device, a location of the wearabledevice, and a distance between the wearable device and a body part of auser of the wearable device.

Aspect 118. The method of any of Aspects 116 to 117, further comprising:receiving, from the wearable device, an additional input configured totrigger the electronic device to stop the privacy mode, wherein theadditional input is based on sensor data indicating at least one of atouch signal corresponding to a touch input at the wearable device, alocation of the wearable device corresponding to a location of a bodypart of a user of the wearable device, and a proximity between thewearable device and the body part.

Aspect 119. The method of any of Aspects 102 to 118, further comprising:determining, based on at least one of data from the wearable device anda command from the wearable device, one or more extended reality (XR)inputs to an XR application on the electronic device.

Aspect 120. The method of Aspect 119, wherein the one or more XR inputscomprise at least one of a modification of a virtual element alongmultiple dimensions in space, a selection of the virtual element, anavigation event, and a request to measure a distance defined by atleast one of the first position of the wearable device, the secondposition of the wearable device, and the movement of the wearabledevice.

Aspect 121. The method of Aspect 120, wherein the virtual elementcomprises at least one of a virtual object rendered by the electronicdevice, a virtual plane in an environment rendered by the electronicdevice, and the environment rendered by the electronic device.

Aspect 122. The method of any of Aspects 120 to 121, wherein thenavigation event comprises at least one of scrolling rendered contentand moving from a first interface element to a second interface element.

Aspect 123. The method of any of Aspects 102 to 122, further comprising:receiving, from the wearable device, an input configured to trigger anadjustment of one or more XR operations at the electronic device,wherein the one or more XR operations comprise at least one of objectdetection, object classification, object tracking, pose estimation, andshape estimation.

Aspect 124. The method of any of Aspects 102 to 123, wherein thewearable device comprises a bracelet, a ring, or a glove, and whereinthe position information comprises at least one of a measurement from aninertial measurement unit from one or more sensors on the wearabledevice and an elevation measured by a pressure sensor from the one ormore sensors.

Aspect 125. A non-transitory computer-readable medium having storedthereon instructions which, when executed by one or more processors,cause the one or more processors to perform a method according to any ofAspects 102 to 124.

Aspect 126. An apparatus comprising means for performing a methodaccording to any of Aspects 102 to 124.

Aspect 127. The apparatus of Aspect 126, wherein the apparatus comprisesa mobile device.

Aspect 128. The apparatus of any of Aspects 126 to 127, wherein theapparatus comprises a camera.

Aspect 129. The apparatus of any of Aspects 126 to 128, wherein theapparatus comprises an XR device and a display.

What is claimed is:
 1. An apparatus comprising: memory; and one or moreprocessors coupled to the memory, the one or more processors beingconfigured to: determine a first position of a wearable device in aphysical space; receive, from the wearable device, position informationassociated with the wearable device; determine a second position of thewearable device based on the received position information; track, basedon the first position and the second position, a movement of thewearable device relative to the apparatus; determine, based on at leastone of the second position of the wearable device or the movement of thewearable device, that the wearable device is outside of a field-of-view(FOV) of one or more image sensors on the apparatus and the movement ofthe wearable device is towards an area within the FOV of the one or moreimage sensors; and initiate, based on a determination that the wearabledevice is outside of the FOV of the one or more image sensors and themovement of the wearable device is towards the area within the FOV ofthe one or more image sensors, one or more imaging operations and one ormore tracking operations at the apparatus, the one or more trackingoperations being at least partly based on image data from the one ormore imaging operations.
 2. The apparatus of claim 1, wherein, to trackthe movement of the wearable device, the one or more processors areconfigured to: determine the first position of the wearable devicewithin a first coordinate system of the wearable device; transform thefirst coordinate system of the wearable device to a second coordinatesystem of the apparatus; and determine the second position of thewearable device within the second coordinate system of the apparatus. 3.The apparatus of claim 1, wherein the one or more processors areconfigured to: track, based on at least one of the second position ofthe wearable device or the movement of the wearable device, a locationof a hand associated with the wearable device.
 4. The apparatus of claim3, wherein the one or more processors are configured to: based on adetermination that the wearable device is within the FOV of the one ormore image sensors, capture one or more images of the hand via at leastone image sensor from the one or more image sensors; and track thelocation of the hand further based on the one or more images of thehand, the location of the hand being tracked relative to a firstcoordinate system of the wearable device.
 5. The apparatus of claim 1,wherein the one or more processors are configured to: based on a firstdetermination that the wearable device is within a first FOV of a firstimage sensor on the apparatus, adjust a first setting of the first imagesensor, the first setting comprising at least one of a power mode of thefirst image sensor or an operating state of the first image sensor; andbased on second determination that the wearable device is outside of asecond FOV of a second image sensor on the apparatus, adjust a secondsetting of the second image sensor, the second setting comprising atleast one of a power mode of the second image sensor or an operatingstate of the second image sensor.
 6. The apparatus of claim 5, wherein,to adjust the first setting of the first image sensor, the one or moreprocessors are configured to change at least one of the power mode ofthe first image sensor from a first power mode to a second power modecomprising a higher power mode than the first power mode or theoperating state of the first image sensor from a first operating stateto a second operating state comprising a higher operating state than thefirst operating state, the second operating state comprising at leastone of a higher framerate or a higher resolution.
 7. The apparatus ofclaim 5, wherein, to adjust the second setting of the second imagesensor, the one or more processors are configured to change at least oneof the power mode of the second image sensor from a first power mode toa second power mode comprising a lower power mode than the first powermode or the operating state of the second image sensor from a firstoperating state to a second operating state comprising a lower operatingstate than the first operating state, the second operating statecomprising at least one of a lower framerate or a lower resolution. 8.The apparatus of claim 1, wherein the one or more processors areconfigured to: based on the determination that the wearable device isoutside of the FOV of the one or more image sensors, track a location ofthe wearable device based on additional position information from thewearable device.
 9. The apparatus of claim 1, wherein the one or moreprocessors are configured to: in response to a determination that thewearable device is within the FOV of the one or more image sensors and aview of the one or more image sensors to the wearable device isobstructed, track a location of the wearable device based on additionalposition information received from the wearable device.
 10. Theapparatus of claim 9, wherein the one or more processors are configuredto: in response to the determination that the wearable device is withinthe FOV of the one or more image sensors and the view of the one or moreimage sensors to the wearable device is obstructed, initialize the oneor more image sensors.
 11. The apparatus of claim 1, wherein, todetermine the first position of the wearable device, the one or moreprocessors are configured to: receive at least one of image data fromone or more image sensors on the apparatus or data associated with oneor more measurements from one or more sensors on the wearable device;and determine the first position of the wearable device based on atleast one of the image data from the one or more image sensors or dataassociated with the one or more measurements from the one or moresensors.
 12. The apparatus of claim 11, wherein the data comprises atleast one of a distance of the wearable device relative to one or moreobjects, a velocity vector indicating a velocity of the movement of thewearable device, a touch signal measured by a touch sensor from the oneor more sensors, audio data from an audio sensor from the one or moresensors, and an elevation of the wearable device in the physical space,and wherein the one or more objects comprises at least one of theapparatus, a body part associated with a user of the wearable device, oran input device.
 13. The apparatus of claim 1, wherein the one or moreprocessors are configured to: determine, based on at least one of datafrom the wearable device or a command from the wearable device, one ormore extended reality (XR) inputs to an XR application on the apparatus.14. The apparatus of claim 13, wherein the one or more XR inputscomprise at least one of a modification of a virtual element alongmultiple dimensions in space, a selection of the virtual element, anavigation event, or a request to measure a distance defined by at leastone of the first position of the wearable device, the second position ofthe wearable device, or the movement of the wearable device.
 15. Theapparatus of claim 14, wherein the virtual element comprises at leastone of a virtual object rendered by the apparatus, a virtual plane in anenvironment rendered by the apparatus, or the environment rendered bythe apparatus.
 16. The apparatus of claim 14, wherein the navigationevent comprises at least one of scrolling rendered content or movingfrom a first interface element to a second interface element.
 17. Theapparatus of claim 1, wherein the one or more processors are configuredto: receive, from the wearable device, an input configured to trigger anadjustment of one or more XR operations at the apparatus, wherein theone or more XR operations comprise at least one of object detection,object classification, object tracking, pose estimation, or shapeestimation.
 18. The apparatus of claim 1, wherein the wearable devicecomprises a bracelet, a ring, or a glove, and wherein the positioninformation comprises at least one of a measurement from an inertialmeasurement unit from one or more sensors on the wearable device or anelevation measured by a pressure sensor from the one or more sensors.19. The apparatus of claim 1, wherein the apparatus comprises a mobiledevice.
 20. The apparatus of claim 19, wherein the apparatus comprises acamera.
 21. The apparatus of claim 19, wherein the apparatus comprisesan XR device including a display.
 22. A method comprising: determining afirst position of a wearable device in a physical space; receiving, fromthe wearable device, position information associated with the wearabledevice; determining a second position of the wearable device based onthe received position information; tracking, based on the first positionand the second position, a movement of the wearable device relative toan electronic device; determining, based on at least one of the secondposition of the wearable device or the movement of the wearable device,that the wearable device is outside of a field-of-view (FOV) of one ormore image sensors on the electronic device and is moving towards anarea within the FOV of the one or more image sensors; and initiating,based on determining that the wearable device is outside of the FOV ofthe one or more image sensors and is moving towards the area within theFOV of the one or more image sensors, one or more imaging operations andone or more tracking operations at the electronic device, the one ormore tracking operations being at least partly based on image data fromthe one or more imaging operations.
 23. The method of claim 22, whereintracking the movement of the wearable device further comprises:determining the first position of the wearable device within a firstcoordinate system of the wearable device; transforming the firstcoordinate system of the wearable device to a second coordinate systemof the electronic device; and determining the second position of thewearable device within the second coordinate system of the electronicdevice.
 24. The method of claim 22, further comprising: tracking, basedon at least one of the second position of the wearable device or themovement of the wearable device, a location of a hand associated withthe wearable device.
 25. The method of claim 24, further comprising:based on a determination that the wearable device is within the FOV ofthe one or more image sensors, capturing one or more images of the handvia at least one image sensor from the one or more image sensors; andtracking the location of the hand further based on the one or moreimages of the hand, the location of the hand being tracked relative to afirst coordinate system of the wearable device.
 26. The method of claim22, further comprising: based on a first determination that the wearabledevice is within a first FOV of a first image sensor on the electronicdevice, adjusting a first setting of the first image sensor, the firstsetting comprising at least one of a power mode of the first imagesensor and an operating state of the first image sensor; and based onsecond determination that the wearable device is outside of a second FOVof a second image sensor on the electronic device, adjusting a secondsetting of the second image sensor, the second setting comprising atleast one of a power mode of the second image sensor and an operatingstate of the second image sensor.
 27. The method of claim 26, whereinadjusting the first setting of the first image sensor further compriseschanging at least one of the power mode of the first image sensor from afirst power mode to a second power mode comprising a higher power modethan the first power mode or the operating state of the first imagesensor from a first operating state to a second operating statecomprising a higher operating state than the first operating state, thesecond operating state comprising at least one of a higher framerate ora higher resolution.
 28. The method of claim 26, wherein adjusting thesecond setting of the second image sensor further comprises changing atleast one of the power mode of the second image sensor from a firstpower mode to a second power mode comprising a lower power mode than thefirst power mode or the operating state of the second image sensor froma first operating state to a second operating state comprising a loweroperating state than the first operating state, the second operatingstate comprising at least one of a lower framerate or a lowerresolution.
 29. The method of claim 22, further comprising: based ondetermining that the wearable device is outside of the FOV of the one ormore image sensors, tracking a location of the wearable device based onadditional position information from the wearable device.
 30. The methodof claim 22, further comprising: in response to a determination that thewearable device is within the FOV of the one or more image sensors and aview of the one or more image sensors to the wearable device isobstructed, tracking a location of the wearable device based onadditional position information from the wearable device.
 31. The methodof claim 30, further comprising: in response to the determination thatthe wearable device is within the FOV of the one or more image sensorsand the view of the one or more image sensors to the wearable device isobstructed, initializing the one or more image sensors.
 32. The methodof claim 22, wherein determining the first position of the wearabledevice further comprises: receiving at least one of image data from oneor more image sensors on the electronic device or data associated withone or more measurements from one or more sensors on the wearabledevice; and determining the first position of the wearable device basedon at least one of the image data from the one or more image sensors ordata associated with the one or more measurements from the one or moresensors, wherein the data comprises at least one of a distance of thewearable device relative to one or more objects, a velocity of themovement of the wearable device, a touch signal measured by the one ormore sensors, audio data from the one or more sensors, and an elevationof the wearable device in the physical space, and wherein the one ormore objects comprise at least one of the electronic device, a body partassociated with a user of the wearable device, or an input device. 33.The method of claim 22, further comprising: determining, based on atleast one of data from the wearable device or a command from thewearable device, one or more extended reality (XR) inputs to an XRapplication on the electronic device, wherein the one or more XR inputscomprise at least one of a modification of a virtual element alongmultiple dimensions in space, a selection of the virtual element, anavigation event, or a request to measure a distance defined by at leastone of the first position of the wearable device, the second position ofthe wearable device, or the movement of the wearable device.
 34. Themethod of claim 33, wherein the virtual element comprises at least oneof a virtual object rendered by the electronic device, a virtual planein an environment rendered by the electronic device, and the environmentrendered by the electronic device, and wherein the navigation eventcomprises at least one of scrolling rendered content or moving from afirst interface element to a second interface element.